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ABSTRACT 


Homeland  Security  Presidential  Directive  #8  led  to  the  establishment  of 
the  National  Exercise  Program  and  the  Top  Officials  exercise  series  to  test  and 
evaluate  first  response  agency  integration  and  effectiveness.  The  last  TOPOFF 
exercise  cost  $16M  and  involved  over  10,000  people,  but  did  not  effectively 
leverage  simulation  techniques  to  make  efficient  use  of  resources. 

This  research  adapts  an  existing  organizational  learning  process, 
integrating  low-  and  high  resolution  simulation  to  provide  decision  support.  This 
process  led  to  the  development  of  a  multi-agent  simulation  methodology  for 
emergency  first  response,  specifically  applied  to  analyze  a  notional  vehicle  bomb 
attack  during  a  festival  in  the  Baltimore  Inner  Harbor. 

This  simulation  demonstrates  the  potential  benefits  of  low  resolution 
simulation,  using  efficient  experimental  design  and  high-performance  computing. 
Combined,  these  two  ideas  result  in  examining  a  48-dimensional  response 
surface  and  using  over  156  CPU  centuries  of  computer  time.  All  experiments 
were  completed  in  less  than  three  weeks. 

The  analysis  of  this  data  set  provided  insight  into  several  areas,  including 
the  importance  of  standing  operating  procedures  in  the  early  moments  of  a  crisis. 
Analysis  showed  that  effective  procedures  may  even  be  more  important  than  the 
effectiveness  of  communications  devices  early  in  a  first  response  operation. 
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EXECUTIVE  SUMMARY 


Large-scale  disasters  can  quickly  overwhelm  the  capabilities  of  state  and 
local  governments.  An  effective  response  in  these  situations  results  from 
integrating  state  and  local  agencies  with  their  federal  counterparts,  thus  enabling 
the  flow  of  needed  resources  and  knowledge.  Toward  this  end,  a 
Presidential  Directive  was  issued  as  part  of  a  plan  to  prepare  for  and  mitigate  the 
effects  of  crisis  events.  This  directive  led  to  the  establishment  of  the  National 
Exercise  Program  (NEP).  National-level  exercises,  such  as  those  that  comprise 
the  NEP,  test  and  evaluate  federal,  state,  and  local  (FSL)  integration  and 
readiness  to  confront  a  manmade  or  natural  disaster. 

Top  Officials  (TOPOFF)  exercises  are  the  foundation  of  the  NEP.  These 
large-scale  exercises  involve  participation  from  all  levels  of  governmental  and 
nongovernmental  agencies  inside  and  outside  the  United  States.  These 
exercises  are  currently  being  planned  and  executed  with  very  little  consideration 
given  to  the  value  of  simulation  as  a  preparation  tool. 

Simulation  is  a  widely  used  decision  support  tool  because  it  allows  staffs 
and  decision  makers  to  explore  given  problems  in  ways  that  are  otherwise 
impractical  (e.g.,  due  to  resources  needed)  or  impossible  (e.g.,  running  an 
exercise  with  thousands  of  parameter  permutations).  The  Department  of 
Homeland  Security  (DHS)  recognizes  the  value  of  simulation  and  reviews 
simulation  models  for  applicability  before  each  TOPOFF  planning  process 
begins.  As  yet,  DHS  has  not  found  the  right  tool  for  the  job. 

Figure  ESI  demonstrates  an  organizational  learning  process  adapted 
from  a  methodology  developed  by  the  U.S.  Army  Training  and  Doctrine 
Command  Analysis  Center  -  Monterey  (TRAC-MTRY);  this  is  the  right  tool  for 
the  job.  This  methodology  is  an  iterative  process  that  uses  a  quick  turnaround, 
low  resolution  model  to  provide  initial  insights  into  a  given  problem.  Those 
insights  are  used  in  the  execution  of  a  high  resolution  simulation,  such  as  a 
wargame.  As  with  any  high  resolution  simulation,  wargaming  results  can  be 
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actionable  results;  that  is,  the  decision  maker  can  use  these  results  to  finalize  the 
plan  that  was  wargamed.  However,  the  decision  maker  can  also  decide  to  adjust 
the  low  resolution  simulation  and  iterate  the  process  until  they  obtain 
satisfactory  results. 


Figure  ESI.  Adaptation  of  TRAC-MTRY's  Learning  Methodology 

The  process  established  in  this  research  expands  the  TRAC-MTRY 
methodology,  which  includes  a  general  flow  that  uses  the  power  of  simulation  to 
train  a  given  audience.  This  research  shows  a  specific  process  by  which 
low  resolution  and  high  resolution  simulation  can  be  used  together  to  help 
organizations  prepare  for  a  TOPOFF  exercise,  or  any  other  large-scale 
training  exercise. 

High-resolution  simulations,  such  as  wargames,  are  established 
decision  support  tools.  TRAC-White  Sands  Missile  Range  (WSMR)  developed  a 
model  to  facilitate  the  execution  of  wargames  in  a  first-response  setting.  The 
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Emergency  Preparedness  incident  Command  System  (EPiCS)  was  used  in 
February  2006  to  simulate  emergency  first  response  to  a  bomb  attack  in 
Baltimore’s  Inner  Harbor  area,  during  a  festival.  To  assess  and  demonstrate  the 
potential  of  an  agile,  low  resolution  simulation  in  this  methodology,  a  multi-agent 
simulation  (MAS)  was  developed  to  simulate  the  same  vignette.  The  simulation 
involves  a  small  terrorist  cell  that  detonates  a  car  bomb,  then  works  to  further 
incite  panic,  while  a  gunman  lies  in  wait  for  first  responders  to  attack.  Police,  fire, 
and  medical  personnel  respond  to  the  bomb  blast  area,  in  which  walking 
wounded  and  stretcher  wounded  civilians  are  panicking.  First  responders  have 
the  following  priorities  of  work,  in  which  police,  Emergency  Medical  Technicians 
(EMTs),  and  firemen  will: 

•  Stabilize  wounded  civilians 

•  Restore  calm  in  the  area 

•  Eliminate  further  threats 

•  Maintain  safety  of  first  responders 


Figure  ES2  is  an  illustrated  screen  shot  of  the  emergency 
first  response  MAS. 


EMTs 

Follow  on  from 

(Mercy  Medical) 

Central  District  &  SWAT 

Diamond  =  ambulatory  wounded 
“X”  =  Stretcher  wounded 


(c> 


Terrorist  Gunman 


Figure  ES2.  Screen  Shot  of  Emergency  First  Response  MAS 
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In  the  analysis  of  the  first  responder  effectiveness,  this  research  involved 
exploring  a  48-dimensional  space  to  gain  an  understanding  of  the  complex 
relationships  involved  in  this  problem.  This  exploration  required  an  efficient 
design  of  experiments.  A  traditional  gridded  design  would  have  resulted  in 
experimental  runs  that  lasted  116  trillion  times  the  current  age  of  the  universe. 
The  design  of  experiments  developed  for  this  research  uses  both  the  flexibility  of 
Flexible  Random  Latin  Hypercube  (FRLH)  sampling,  and  the  space  filling  nature 
of  Nearly  Orthogonal  Latin  Hypercube  (NOLH)  sampling.  The  use  of  these 
efficient  designs,  in  conjunction  with  use  of  the  supercomputers  at  the 
Maui  High  Performance  Computing  Center  (MHPCC),  resulted  in  the  use  of 
137,277,343  CPU  hours,  or  156  CPU  centuries.  The  data  set  analyzed  for  this 
research  is  nearly  52,000  rows  by  more  than  5,300  columns. 

The  analysis  of  the  data  from  this  model  suggest: 

•  Overwhelmingly,  the  most  important  factor  in  achieving  success  in 
crisis  mitigation  is  the  effectiveness  of  the  police  in  taking  positive 
control  of  the  crowd,  exerting  calming  influence,  and  providing 
direction. 

•  If  a  police  force  is  not  well  trained,  and  therefore  not  very  effective, 
the  officers  may  achieve  greater  success  by  being  less  persistent 
with  individuals;  that  is,  by  spreading  their  influence  more  broadly. 

•  Well  established,  well  executed  standing  operating  procedures 
(SOPs)  may  play  a  more  important  role  in  first  response  operations 
than  interagency  communication. 

•  There  may  be  a  level  of  diminishing  returns  for  first  responder 
training;  that  is,  a  person  can  be  only  so  trained.  After  that  level  is 
reached,  it  may  be  more  effective  to  leverage  resources  elsewhere. 

Results  of  the  data  analysis  are  not  meant  to  directly  apply  to  actual 
emergency  response  techniques,  or  specifically  to  the  City  of  Baltimore.  This 
model  does  not  include  the  actual  force  structure  and  SOPs  from  Baltimore,  but 
data  adapted  from  the  February  2006  EPiCS  run.  This  research  is  a  proof  of 
concept  to  show  that  it  is  possible  to  quickly  and  credibly  model  emergency 
first  response  with  a  MAS,  and  the  data  analysis  from  such  a  credible,  verified, 
and  calibrated  model  will  be  useful  and  insightful. 
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The  single  most  important  result  of  this  research  comes  not  from  the  data 
analysis,  but  from  the  developed  methodologies.  Simulation  is  a  decision 
support  technique  that  is  relevant  to  emergency  preparedness,  especially  to  an 
exercise  program  the  size  and  complexity  of  the  TOPOFF  program.  The 
organizational  learning  technique  discussed  herein  and  the  incorporation  of  MAS 
in  emergency  first  response  simulation  can  help  train  first  response  organizations 
more  effectively,  resulting  in  better  crisis  mitigation  and  lives  saved. 
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I.  INTRODUCTION 


Our  Federal,  State,  and  local  governments  would  ensure  that  all 
response  personnel  and  organizations — including  law  enforcement, 
military,  emergency  response,  health  care,  public  works,  and 
environmental  communities — are  properly  equipped,  trained,  and 
exercised  to  respond  to  all  terrorist  threats  and  attacks  in  the 
United  States.1 


National  Strategy  for  Homeland  Security 

July  2002 

A.  NATIONAL  EXERCISE  PROGRAM  BACKGROUND 

The  President  issued  Homeland  Security  Presidential  Directive  Number  8 
(HSPD-8,  Subject:  National  Preparedness)  as  part  of  a  plan  to  prepare  for  and 
mitigate  the  effects  of  crisis  events.  In  HSPD-8,  the  President  directed  the 
Secretary  of  the  Department  of  Homeland  Security  (DHS)  to 

.  .  .  establish  a  national  program  and  a  multi-year  planning  system 
to  conduct  homeland  security  preparedness-related  exercises  that 
reinforces  identified  training  standards,  provides  for  evaluation  of 
readiness,  and  supports  the  national  preparedness  goal.2 

This  presidential  directive  led  to  the  establishment  of  the  National  Exercise 
Program  (NEP).  National-level  exercises,  such  as  those  that  comprise  the  NEP, 
evaluate  federal,  state,  and  local  (FSL)  integration  and  test  “collective 
preparedness,  interoperability,  and  collaboration  across  all  levels  of  government 
and  the  private  sector.”3 

State  and  local  preparedness  is  necessary,  but  insufficient,  for  effective 
disaster  response,  as  large-scale  disasters  can  quickly  overwhelm  the 


1 Homeland  Security  Exercise  and  Evaluation  Program,  Volume  I:  Overview  and  Doctrine, 
revised  May  2004,  p.  4.  Retrieved  on  26  January  2006  from  the  World  Wide  Web  at 
http://www.oip.usdoi.gov/odp 

2“Homeland  Security  Presidential  Directive  #8.”  Retrieved  on  5  December  2005  from  the 
World  Wide  Web  at  http://www.whitehouse.gov/news/releases/2003/12/20031217-6.html 
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capabilities  of  state  and  local  governments.  Effective  response  in  these 
situations  results  from  integrating  state  and  local  agencies  with  their  federal 
counterparts,  thus  enabling  the  flow  of  needed  resources  and  knowledge. 

The  NEP  is  one  component  of  the  DHS’s  endeavor  to  achieve  the 
National  Preparedness  Goal  (NPG),  established  by  HSPD-8.  The  NPG 
“envisions  a  national  system  in  which  all  agencies  have  the  capabilities  they 
need  to  effectively  communicate  and  coordinate  resources.”4  The  following 
seven  measurable  priorities  quantify  the  level  of  FSL  preparedness  and 
achievement  of  the  NPG. 

Overarching  Priorities.  Three  NPG  priorities  contribute  to  the 
development  of  several  capabilities: 

•  Implement  the  National  Incident  Management  System  (NIMS)  and 
National  Response  Plan  (NRP) 

•  Expand  regional  collaboration 

•  Implement  the  Interim  National  Infrastructure  Protection  Plan 

Capability-Specific  Priorities.  Four  NPG  priorities  are  capability- 
specific,  providing  specific  means  that  the  nation  needs  most: 

•  Strengthen  information  sharing  and  collaboration  capabilities 

•  Strengthen  interoperable  communications  capabilities 

•  Strengthen  chemical,  biological,  radiological,  nuclear,  and  explosive 
(CBRNE)  detection,  response,  and  decontamination  capabilities 

•  Strengthen  medical  surge  and  prophylaxis  capabilities  to  enable 
emergency-ready  public  health  and  medical  facilities  across 
the  nation5 

Four  of  the  seven  NPG  priorities  address  the  need  for  efficient  and 
effective  command  and  control,  communication,  and  resource  sharing  in  crisis 
events.  The  NEP  established  the  Top  Officials  (TOPOFF)  exercise  as  their 

4“Fact  Sheet:  The  National  Priorities,”  Department  of  Homeland  Security.  Retrieved  on 

5  December  2005  from  the  World  Wide  Web  at  http://www.oip.usdoi.gov/odp/docs/ 
Priorities  041305.pdf 
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cornerstone  event  to  assess  the  progress  of  DHS  in  meeting  its 
NPG  priorities. 

TOPOFF  is  a  biennial  series  of  exercises  mandated  by  Congress  to 
strengthen  the  nation’s  crisis  response.* * 6  These  exercises  have  grown  in  size 
and  scope  since  the  first  one  in  2000.  TOPOFF  2000  was  a  $3.5  million  national 
event,  incorporating  “numerous”  FSL  agencies.7  TOPOFF  3,  the  most  recent 
exercise,  was  a  $16  million  international  event  with  over  200  FSL  agencies  and 
10,000  participants.8  The  resources  dedicated  to  this  exercise  series  help 
demonstrate  the  importance  of  this  program  and  its  role  in  developing  crisis 
response  capabilities.  This  research  will  support  the  TOPOFF  program  by 
helping  the  exercise  planning  and  execution  process  mature  to  more  efficiently 
and  effectively  meet  the  NPG. 

B.  SIMULATION  AS  A  TOPOFF  FACILITATOR 

Simulation  is  one  of  the  most  widely  used  operations  research  and 

management  science  techniques,  if  not  the  most  widely  used.9 

Law  and  Kelton,  2000 

Simulation  is  a  widely  used  decision  support  tool  because  it  enables  an 
organization’s  staff  to  efficiently  examine  problems  that  would  otherwise  be 
expensive  to  analyze.  The  staff  presents  its  recommendation  to  the 
decision  maker  based  on  analysis  of  the  simulation.  For  example,  suppose  a 


department  of  Homeland  Security,  “National  Exercise  Program  Overview,”  Office  for 

Domestic  Preparedness.  Retrieved  on  1  December  2005  from  the  World  Wide  Web  at 

http://www.oip.usdoi.gov/odp/exercises.htm 

7U.S.  National  Response  Team,  “Exercise  TOPOFF  2000  and  National  Capital  Region  After 
Action  Report,”  2001.  Retrieved  on  5  December  2005  from  the  World  Wide  Web  at 
http://www.nrt.org/Production/NRT/NRTWeb.nsf/PaqesBvLevelCat/Level3TOPOFF? 

Opendocument 

department  of  Homeland  Security,  Press  Room,  “TOPOFF  3  Frequently  Asked  Questions.” 
Retrieved  on  2  March  2006  from  the  World  Wide  Web  at  http://www.dhs.gov/ 
dhspublic/interapp/editorial/editorial  0603.xml. 

9Averill  M.  Law,  and  W.  David  Kelton,  Simulation  Modeling  and  Analysis ,  3rd  Edition,  2000, 
P-2. 
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manufacturing  company  is  considering  the  construction  of  an  extension  to  one  of 
its  plants.  The  company  will  not  build  the  extension,  track  change  in  profit,  then 
tear  down  the  extension  if  it  results  in  a  net  loss.  However,  the  company  may 
conduct  a  form  of  simulation  to  examine  the  likely  change  in  profit  based  on  the 
decision  to  build.  Using  this  analysis,  a  recommendation  is  made  to  the 
decision  maker  to  build  or  not  to  build.10 

Simulation  is  also  a  useful  decision  support  technique  for  analyzing  the 
execution  of  contingency  plans.  The  U.S.  military  frequently  wargames  its 
military  plans  before  the  execution  of  the  plan.  Wargaming  is  a  “human-in-the- 
loop”  simulation.  Human-in-the-loop  means  that  people  are  actually  involved, 
making  decisions,  throughout  the  simulation.  In  wargaming,  a  plan  is  analyzed 
by  examining  an  event  in  terms  of  a  side’s  action,  another  side’s  counteraction, 
and  the  first  side’s  subsequent  counteraction,  and  so  on.  Through  analyzing 
both  the  results  of  the  simulation  and  the  process  by  which  those  results  were 
achieved,  a  staff  can  recommend  a  plan  to  the  decision  maker  for  approval. 

In  addition  to  its  usefulness  in  determining  the  allocation  of  resources  and 
analyzing  contingency  planning,  simulations  are  good  training  tools. 
Human-in-the-loop  simulations,  such  as  wargames,  provide  the  target  audience 
with  a  chance  to  react  to  situations  they  may  encounter  when  the  actual  event 
occurs.  Computer  simulations  usually  run  more  quickly  than  human-in-the-loop 
simulations,  and  can  result  in  many  more  iterations  being  run.  These  many 
iterations  can  provide  insightful  answers  to  “what  if  questions  not  able  to  be 
answered  in  a  slower-running  human-in-the-loop  simulation. 

The  DHS’s  Office  of  State  and  Local  Government  Coordination  and 
Preparedness  (SLGCP)  recognizes  the  potential  value  of  simulation  in 
preparation  for  the  TOPOFF  program.  Prior  to  TOPOFF  3,  DHS  reviewed 
100  models,  simulations,  and  games  (MS&G),  evaluating  their  usefulness  in 
preparation  for  and  execution  of  the  exercise.  The  result  of  this  review  was  that 

10Averill  M.  Law,  and  W.  David  Kelton,  Simulation  Modeling  and  Analysis ,  3rd  Edition, 

2000,  p.  1. 
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some  MS&Gs  proved  useful,  but  “no  single  group  of  products  can  be 
recommended  that  will  support  the  requirements  of  all  communities.”11 

The  Training  and  Doctrine  Command  Analysis  Center-Monterey 
(TRAC-MTRY)  is  currently  studying  the  use  of  simulation  techniques  to  facilitate 
organizational  learning.  Figure  1  illustrates  a  methodology  proposed  by 
TRAC-MTRY  to  improve  organizational  learning,  especially  in  support  of  the 
TOPOFF  program.  This  iterative  methodology  includes  scenario  planning  as  the 
first  step.  An  analyst  models  the  scenario  with  a  high  resolution  simulation,  to  be 
used  as  a  training  tool  for  exercise  participants.  Data  that  results  from  the 
simulation  are  analyzed  and  the  base  simulation  adjusted  as  necessary  to 
correct  modeling  deficiencies  or  perform  “what  if  analyses.  If  decision  makers 
are  satisfied  with  the  training  achieved,  the  exercise  can  be  executed  and 
results  documented. 


Figure  1.  TRAC-MTRY  Proposed  TOPOFF  Methodology  to  Improve  the 

Learning  Process12 


^ThoughtLink,  Inc.  “Review  of  Models,  Simulations,  and  Games  for  Domestic  Exercises  and 
Preparedness,”  (2004).  Retrieved  on  5  December  2005  from  the  World  Wide  Web  at 
http://www.oip.usdoi.gov/odp/docs/Review  of  MSG  SlimVersion.pdf 
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Figure  2  illustrates  a  revised  methodology,  adapted  from  the  TRAC-MTRY 
organizational  learning  methodology.  The  following  paragraphs  are  numbered  in 
accordance  with  the  steps  in  this  methodology. 


Figure  2.  Adaptation  of  TRAC-MTRY's  Learning  Methodology13 

1.  Model  Scenarios  with  Multi-Agent  Simulation  (MAS) 

MAS  are  low  resolution  models  that  are  (relatively)  easy  to  set  up  and 
alter,  and  are  fast  to  run.  Although  it  is  not  usually  appropriate  to  use  MAS  to 
model  an  entire  TOPOFF  exercise,  it  is  useful  to  model  specific  vignettes  from 
the  exercise  plan.  The  power  of  MAS  is  enabled  by  the  use  of  data  farming  to 


12Doris  M.  Turnage  and  LTC  Jeffrey  B.  Schamburg,  Facilitating  Organizational  Learning  and 
Change  Through  the  National  Exercise  Program,  (December  2005). 

13lbid. 
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thoroughly  explore  the  range  of  possibilities  in  these  vignettes.  The  analyst 
leverages  modern  computing  and  supercomputing  power  to  vary  parameters  and 
run  the  scenario  thousands  of  times. 

2.  Conduct  Training  or  Analysis  Runs 

MAS  can  be  used  for  a  variety  of  purposes.  Within  the  context  of  this 
research,  there  are  two  major  applications  for  low  resolution  simulation.  First,  the 
MAS  may  be  used  to  train  leaders  and  staffs.  Inputting  the  scenario  into  the 
model  and  visualizing  the  events  that  unfold  during  a  given  model  run  can  help  to 
validate  and  streamline  procedures.  Second,  data  collection  and  the  subsequent 
analysis  can  provide  decision  makers  with  insight  into  “what  if  questions  that 
would  be  difficult  to  answer  in  another  setting. 

3.  Collect  MAS  Lessons  Learned 

The  data  farming  approach  to  analysis  leads  to  a  much  more  thorough 
exploration  of  the  solution  space  than  more  complex,  slower-running, 
high  resolution  models.  By  understanding  the  solution  space’s  “landscape,”  we 
are  able  to  consider  “what  if  questions  and  gain  unique  insight  into  the 
problem.14  Analysts  may  use  the  insights  gained  from  output  analysis  in  two 
possible  ways.  The  output  may  indicate  a  need  to  refine  the  MAS  scenario,  or 
rerun  the  experimental  design  to  focus  on  a  certain  area.  Alternatively,  if  the 
analyst  is  satisfied  with  the  results  of  the  data  analysis,  the  insights  are  captured 
as  lessons  learned.  The  lessons  learned  from  the  low  resolution  simulation  are 
used  as  a  starting  point  for  a  high  resolution  simulation,  such  as  a 
wargame  simulation. 

4.  Develop  Wargame  Model  Using  MAS  Lessons  Learned 

MAS  lessons  learned  can  help  provide  insight  into  making  necessary 
assumptions  as  the  model  is  developed.  During  the  execution  of  the 

14LTC  Thomas  M.  Cioppa,  “A  Potential  Role  of  Agent  Based  Models  in  Military  Analysis,” 
Information  Paper,  (prepared  December  2003). 
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human-in-the-loop  wargame,  MAS  lessons  learned  can  assist  decision  makers  in 
making  effective,  timely  decisions. 

5.  Conduct  Wargame  Incorporating  MAS  Lessons  Learned 

The  wargame  can  be  either  a  timestep  simulation  or  event-driven. 
Common  to  both  approaches  is  that  one  side  will  act,  the  other  will  react,  and  the 
first  side  will  then  counteract,  and  then  move  to  the  next  timestep  or  event.  This 
type  of  simulation  is  high  resolution,  meaning  that  the  output  is  more 
representative  of  actual  events  than  the  output  from  a  low  resolution  simulation, 
and  thereby  more  actionable.15 

TRAC-White  Sands  Missile  Range  (WSMR)  developed  the  Emergency 
Preparedness  incident  Command  System  (EPiCS),  which  is  a  computer-based, 
high  resolution  simulation.  EPiCS  is  a  version  of  the  Joint  Army  Navy  Uniform 
Simulation  (JANUS),  a  well-known  simulation  used  for  many  years  to  facilitate 
military  wargaming.  TRAC-WSMR  developed  EPiCS  to  facilitate  wargaming  in 
emergency  response  situations  requiring  multi-echelon  and/or  interagency 
communication  and  coordination.16 

6.  Satisfied  With  Results? 

The  decision  maker  must  review  the  products,  insights,  and 
lessons  learned  from  the  MAS  and  wargame.  If  satisfied  that  enough  insight  has 
been  gained  by  the  staff,  the  decision  maker  directs  the  staff  to  document  the 
lessons  learned  and  finalize  the  plan.  The  decision  maker  may  require  more 
information,  different  information,  or  a  narrowly  focused  view  of  a  small  part  of 
the  response  surface.  In  this  case,  the  staff  iterates  the  process  of  MAS 
modeling  and  wargaming. 


15Defense  Modeling  and  Simulation  Office  Verification,  Validation,  and  Accreditation 
Glossary.  Retrieved  on  20  March  2006  from  the  World  Wide  Web  at 

https://www.dmso.mil/public/librarv/proiects/vva/qlossary.pdf 

16EPiCS  homepage.  Retrieved  on  17  May  2006  from  the  World  Wide  Web  at 
http://epics.astcorp.com 
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7.  Document  Wargaming  Lessons  Learned 

Clearly  and  concisely  documenting  the  lessons  learned  from  the  wargame 
is  as  important  as  running  a  realistic  simulation,  and  by  doing  so  throughout  the 
simulation  process,  we  can  disseminate  the  information  to  the  people  that  need 
it.  Wargaming  lessons  learned  may  be  used  in  two  ways:  to  refine  the  original 
MAS  and  then  begin  the  process  again  or  to  finalize  the  exercise  plan. 

Organizational  learning  does  not  stop  with  the  final  simulation;  it  continues 
throughout  the  TOPOFF  exercise,  and  after.  Agency  leaders  and 
first  responders  share  the  knowledge  they  gain  during  the  execution  of  the 
exercise  during  “hot  wash”  after  action  reviews  (AARs).  The  purpose  of  these 
short  AARs  is  to  capture  important  findings  before  too  much  time  passes  and 
memories  fade.  Following  the  TOPOFF  exercise,  a  senior  observer  leads  a 
comprehensive  analysis  of  the  exercise,  to  gain  final  insights  into  strengths  and 
weaknesses  of  the  response.  Action  facilitated  by  these  insights  is  the  beginning 
of  preparation  for  the  next  TOPOFF  exercise  or  next  local  crisis  event. 

C.  PROBLEM  STATEMENT 

The  SLGCP  currently  prepares  for  its  biennial  TOPOFF  exercises  without 
the  use  of  computer  simulation  (e.g.,  MAS),  and  with  limited  use  of 
human-in-the-loop  simulation  (e.g.,  wargaming).  Inclusion  of  simulation  in  the 
planning  process  of  TOPOFF  exercises  could  result  in  increased  preparedness, 
more  efficient  resource  allocation,  and  more  meaningful  lessons  learned.  The 
goal  of  this  research  is  to  provide  a  methodology  for  an  analyst  to  use 
Pythagoras,  or  another  MAS  model,  to  simulate  emergency  response  to  a  natural 
or  man-made  crisis  event.  The  methodology  developed  in  this  thesis  can  be 
applied  in  many  different  situations,  such  as  a  future  TOPOFF  scenario  in  a 
major  city,  an  annual  force  protection  exercise  at  a  military  installation,  or  a  small 
town’s  response  to  a  flood. 
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D.  SCOPE 

This  research  will  focus  on  the  police,  fire,  and  medical  response  to  a 
small  terrorist  cell  detonating  a  bomb  near  the  amphitheater  in  Baltimore’s 
Inner  Harbor.  This  event  replicates  the  event  that  occurred  in  exercise 
TOPOFF  3  in  New  London,  Connecticut.  TRAC-WSMR  also  replicated  the 
New  London  event  within  their  wargaming  simulation,  EPiCS.  TRAC-WSMR 
provided  the  specific  scenario  and  force  structure  information  used  in  the  EPiCS 
run  conducted  in  February  2006.  This  MAS  development  will  replicate  the  first 
hour  of  TRAC-WSMR’s  EPiCS  run  as  closely  as  possible,  using  a 
multi-agent  approach. 

This  work  is  focused  on  the  following  research  questions: 

•  What  is  an  appropriate  methodology  for  use  of  a  MAS  environment 
in  the  modeling  of  emergency  response  to  the  simulated 
vehicle-borne  improvised  explosive  device  (VBIED)  that  explodes 
near  the  amphitheatre  in  Baltimore’s  Inner  Harbor? 

•  What  is  the  most  appropriate  mix  of  police,  fire,  and  medical 
assets? 

•  What  is  the  most  effective  interagency  communication  architecture 
for  emergency  response  to  VBIED  in  Baltimore’s  Inner  Harbor? 

E.  THESIS  OVERVIEW 

The  following  chapters  will  provide  specific  background  on  the  simulation 
area,  details  on  how  the  model  was  constructed,  how  the  data  were  collected 
and  analyzed,  and  the  resultant  conclusions.  Chapter  II  presents  detailed 
information  to  the  reader  that  addresses  the  specific  scenario  analyzed  for  this 
research  and  the  organizations  included  in  the  model.  Chapter  III  addresses  the 
development  and  application  of  a  methodology  for  simulating  emergency 
first  response  in  a  multi-agent  environment.  Chapter  IV  discusses  the 
implementation  of  the  model  in  detail,  discussing  the  model’s  settings, 
deficiencies,  and  limitations.  Chapter  V  details  the  factors  that  are  analyzed  and 
the  experimental  design  methodology.  In  Chapter  VI,  the  author  demonstrates 
the  analysis  of  data  gained  from  executing  the  experimental  design.  The  final 
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chapter  presents  an  analysis  of  the  data  from  the  simulation,  significant  results, 
and  possible  follow-on  work. 
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II.  BALTIMORE  ATTACK  SCENARIO  OVERVIEW 


This  chapter  provides  the  reader  with  detailed  information  about  the 
scenario  and  organizations  studied  as  the  basis  for  this  research.  The  chapter’s 
first  section  outlines  the  TOPOFF  3  exercise,  the  simulated  bomb  attack,  and  the 
reason  for  moving  the  location  of  the  simulation  to  Baltimore  from  New  London 
for  this  research.  The  chapter’s  second  section  provides  information  about  the 
organizations  simulated,  including  force  structure  and  purpose. 

A.  TOPOFF  3  SCENARIO 

The  DHS  conducted  TOPOFF  3  during  April  4-8,  2005.  This  exercise 
simulated  two  coordinated  terrorist  attacks.  The  first  was  a  pneumonic  plague 
attack  in  Union  and  Middlesex  counties,  New  Jersey.  The  second  was  a 
chemical  agent  release  (mustard  gas)  and  high-yield  explosive  in  the 
City  of  New  London,  Connecticut.17 

The  goal  of  this  research  is  to  develop  a  simulation  methodology  to  model 
emergency  first  response  in  a  crisis  situation.  The  methodology  must  be  robust, 
flexible,  and  capable  of  addressing  various  crisis  scenarios  in  differing 
geographical  settings.  The  author  considered  two  ways  to  proceed: 

•  Develop  a  detailed  simulation  and  analysis  of  one  vignette 

•  Develop  and  analyze  several  smaller,  less  detailed  vignettes 

To  create  a  simulation  that  is  realistic  and  widely  applicable,  it  is 
necessary  to  be  detailed  in  the  approach  of  simulation  design  and  analysis.  The 
detailed  approach  enables  the  analyst  to  answer  questions  and  solve  problems 
that  do  not  surface  in  a  more  cursory  approach.  It  is  detailed  thinking  and 
problem-solving  that  makes  the  simulation  methodology  robust.  Therefore,  the 


17Department  of  Homeland  Security,  “A  Review  of  the  Top  Officials  3  Exercise,”  Office  of  the 
Inspector  General,  Office  of  Inspections  and  Special  Reviews,  p.  4.  Retrieved  on 
25  January  2006  from  the  World  Wide  Web  at  http://www.dhs.gov/interweb/assetlibrarv/OIG 
06-07  Nov05.pdf 
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scope  of  this  research  is  limited  to  the  high-yield  explosive  attack,  which  occurs 
during  a  festival  that  draws  approximately  10,000  attendees. 

The  explosion  simulated  during  TOPOFF  3  occurred  on  April  4,  2005, 
during  the  annual  Celebrate  Connecticut  festival  held  on  the  New  London 
waterfront.  A  small  terrorist  cell,  comprised  of  approximately  five  personnel, 
orchestrated  a  VBIED  attack.  The  VBIED  was  the  equivalent  of  7,350  pounds  of 
ammonium  nitrate,  approximately  the  same  size  as  the  bomb  that  destroyed  the 
Murrah  Federal  Office  Building  in  Oklahoma  City.18  Figure  3  illustrates  the 
affected  area. 


Figure  3.  Affected  Area  of  TOPOFF  3  VBIED,  New  London,  CT19 


18E-mail  from  Dr.  Julie  Seton,  EPiCS  Program  Manager,  TRAC-WSMR,  titled  “Message  from 

Dan  Edmonson  -  CT  Incident  Details  for  TRAC-Monterey  Project,”  24  January  2006, 

office  communication. 

19Graphic  retrieved  on  17  May  2006,  Using  Google  Earth;  affected  area  provided  by 
TOPOFF  3  Full  Scale  Exercise  Final  Planning  Conference  Brief,  given  March  2-3,  2005. 

Retrieved  on  13  January  2006  from  the  World  Wide  Web  at 

http://www.dhs.gov/interweb/assetlibrarv/OIG  06-07  Nov05.pdf 
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B.  MOVEMENT  OF  VIGNETTE  TO  BALTIMORE 


This  simulation  setting  was  moved  to  Baltimore  due  primarily  to  funding 
constraints.  The  high  resolution  model  chosen  for  comparison  is 
TRAC-WSMR’s  EPiCS  model.  The  terrain  for  Baltimore  was  available  when  the 
EPiCS  simulation  was  built;  however,  the  terrain  for  New  London  was  not 
available  and  would  have  been  both  time-consuming  and  expensive  to  build. 

Movement  of  the  event  from  its  original  venue  does  not  decrease  this 
research’s  relevance.  Certain  necessary  similarities  exist  between  New  London 
and  Baltimore  that  make  these  cities  an  ideal  pairing.  Both  cities  celebrate 
festivals  that  draw  approximately  10,000  people,  resulting  in  similar  response 
challenges  due  to  crowd  control.  Both  cities  are  located  waterside,  along  an 
extensive  pier  network,  providing  the  requirement  for  similar  maritime  response. 

There  are  advantages  to  simulating  the  event  in  a  different  setting  than  the 
original  exercise.  By  setting  the  simulation  in  Baltimore,  the  results  cannot  be 
artificially  “gamed”  to  look  like  the  results  of  the  exercise  actually  run  during 
TOPOFF  3.  In  addition,  transporting  the  scenario  to  different  terrain  shows  that 
the  methodology  developed  for  simulation  is  applicable  in  different  cities,  not  just 
in  the  place  were  the  event  was  actually  executed. 

C.  DESCRIPTION  OF  FIRST  RESPONDERS 

Three  primary  organizations  provide  emergency  first  response  during  a 
major  civil  crisis  event:  police  departments,  fire  departments,  and  hospitals.  The 
response  from  each  organization  occurs  in  stages,  enabling  these  organizations 
to  provide  immediate  support,  not  needing  to  wait  until  resources  are  assembled 
and  staged.  The  following  section  is  a  synopsis  of  each  organization’s  force 
structure  and  purpose  at  each  phase  of  the  crisis  response  operation. 

The  overall  mission  of  the  response  forces,  in  priority  order,  is: 

•  Control  the  crowd;  localizing  the  bomb’s  effect  and  minimizing 
further  injuries 

•  Identify  and  eliminate  additional  threats 
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•  Identify  the  dead 

•  Triage  and  stabilize  the  wounded 

1.  Initial  Response 

Initial  response  comes  from  the  emergency  services  personnel  that  are  in 
the  immediate  crisis  area  when  the  event  occurs.  The  purpose  of  initial  response 
is  security  and  safety.  The  initial  response  force  simulated  in  this  exercise  is 
comprised  of  police  officers  with  two  different  missions.  Police  on  foot  patrol 
circulate  throughout  the  Celebrate  Baltimore  area.  Traffic  police  maintain 
positions  at  each  intersection  leading  into  and  out  of  the  festival  area. 

Foot  patrols  in  the  Inner  Harbor  area  serve  several  purposes.  Their 
primary  purpose  is  to  provide  police  presence  and  security  in  the  area.  The 
secondary  purposes  are  to  conduct  crowd  control  and  provide  medical 
assistance  as  needed.  Traffic  police  from  several  separate  police  departments 
are  stationed  at  the  intersections  in  the  Celebrate  Baltimore  area.  Their  primary 
purpose  is  crowd  control,  in  addition  to  providing  police  presence  in  the  area. 

There  are  22  police  officers  who  comprise  the  initial  response  police 
presence.  Ten  are  on  foot  patrol  (five  patrols  of  two  officers  each)  and  one  traffic 
officer  stands  at  each  of  12  intersections  (see  Figure  4  for  a  diagram  of  police 
locations).  All  initial  response  police  carry  similar  equipment.  They  are  armed 
with  a  standard  issue  9mm  semiautomatic  pistol  and  wear  Level  lll-A  body 
armor,  protecting  them  against  bullet  penetration  from  most  handguns  and  some 
small  caliber  rifles.  All  police  carry  handheld  radios  capable  of  communicating 
throughout  the  simulation  area.20 

Following  the  bomb  attack,  both  patrol  and  traffic  police  maintain  their  pre¬ 
attack  missions.  The  primary  focus  of  the  police  in  this  situation  is  to  maintain 
crowd  control  to  limit  the  number  of  casualties  caused  by  people  running  away 
from  the  attack  site.  While  maintaining  crowd  control,  police  try  to  identify 
additional  threats  that  may  attempt  secondary  attacks.  Police  on  foot  patrol  will 

on 

E-mail  from  Dr.  Julie  Seton,  EPiCS  Program  Manager,  TRAC-WSMR  titled  “Force  Files,” 
dated  20  January  2006,  office  communication. 
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continue  to  patrol,  attempting  to  calm  as  many  people  as  possible.  Traffic  police 
will  maintain  their  positions  at  intersections,  keeping  nonessential  personnel  out 
of  the  affected  area  in  order  to  minimize  congestion  that  will  hamper 
follow-on  forces. 


Figure  4.  Array  of  Initial  Response  Forces  (Police)21 


2.  Follow-On  Response 

Follow-on  response  is  comprised  of  police,  fire,  and  medical  personnel 
that  are  summoned  to  the  scene  of  the  attack  by  the  central  dispatch  authority. 
The  purpose  of  follow-on  responders  is  to  save  lives,  while  police  continue  to 
provide  crowd  control  and  security.  The  fire  department  controls  the  threat  due 
to  fire  (but  will  also  assist  in  the  establishment  of  triage  points),  takes  the  lead  in 
hazardous  material  situations,  and  provides  medical  assistance  with  its  organic 


21Graphic  retrieved  from  Google  Earth,  12  April  2004;  locations  are  adapted  from 
TRAC-WSMR  EPiCS  simulation  conducted  in  February  2006.  Free  application  downloaded  on 
30  March  2006  from  the  World  Wide  Web  at  http://earth.qooqle.com/download-earth.htm. 
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assets.  Hospitals  provide  doctors,  nurses,  and  support  staff  to  triage,  stabilize, 
and  treat  injured  people  on-site.  In  addition  to  site  support,  hospitals  have  the 
ability  to  provide  surge  support  at  their  location.  Figure  5  illustrates  the  initial 
location  of  follow-on  responders  and  their  distance  from  the  location  of  the 
bomb  blast. 
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Figure  5.  Follow-on  Response  (Police,  Fire,  Medical)22 

Follow-on  police  response  comes  from  the  Central  District 
Police  Headquarters,  approximately  one-third  of  a  mile  from  the  scene  of  the 
explosion.23  Police  officers  are  diverted  from  their  standard  daily  duties  and 
phased  into  the  response.  Follow-on  patrolmen  have  dual  missions:  control  the 


22Background  graphics  and  distances  courtesy  of  Mapquest.com.  Retrieved  on 
30  March  2006  from  the  World  Wide  Web  at  http://www.mapquest.com 

23Location  of  Central  District  Baltimore  Police  Headquarters.  Retrieved  on  30  March  2006 
from  the  World  Wide  Web  at  http://www.ci.baltimore.md.us/neiqhborhoods/facilities/polfire.html 
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crowd  to  minimize  the  effect  of  panic  and  identify  and  eliminate  further  terrorist 
threats.  Providing  basic  first  aid  is  a  tertiary  task. 

Follow-on  police  are  equipped  in  the  same  manner  as  the  initial 
responders.  They  are  armed  with  a  standard  issue  9mm  semiautomatic  pistol 
and  wear  Level  lll-A  body  armor,  protecting  them  against  bullet  penetration  from 
most  handguns  and  some  small  caliber  rifles.  All  police  carry  handheld  radios 
capable  of  communicating  throughout  the  simulation  area. 

In  addition  to  the  diverted  patrols,  the  district’s  two  Special  Weapons  and 
Tactics  (SWAT)  teams  are  located  at  the  Central  District  Headquarters.  The 
SWAT  teams  remain  on  call,  staged  and  ready  at  headquarters,  throughout  the 
simulation.  If  an  armed  terrorist  threat  is  identified,  the  threat  location  is  radioed 
to  the  SWAT  team.  The  SWAT  team  responds,  eliminates  the  threat,  and  returns 
to  the  department  headquarters  to  await  another  call. 

The  SWAT  teams  have  greater  combat  capability  than  the  police  patrols. 
SWAT  team  members  carry  an  M4  carbine,  in  addition  to  the  9mm 
semiautomatic  pistol.  They  also  wear  body  armor  that  is  analogous  to  the  Army’s 
Interceptor  body  armor,  which  is  more  protective  than  the  standard 
Level  lll-A  body  armor  worn  by  patrolmen.  SWAT  team  members  also  carry  a 
handheld  radio  that  can  range  over  the  entire  simulation  area. 

Medical  responders  include  doctors,  nurses,  Emergency  Medical 
Technicians  (EMTs),  and  support  staff  from  three  area  hospitals.  Mercy  Medical 
Center  is  located  approximately  nine-tenths  of  a  mile  north  of  the  explosion. 
University  Hospital  is  about  one  mile  west  of  the  scene.  Saint  Johns  Hopkins 
Hospital  is  approximately  two  miles  northeast.24  Hospitals  respond  to  the 
emergency  with  teams  of  ten,  comprised  of  doctors,  nurses,  support  staff,  and 
associated  equipment.  These  teams  arrive  incrementally,  every  45  minutes, 
after  the  event.  The  primary  purpose  of  the  medical  teams  is  to  triage  the 
wounded,  stabilize  them,  and  evacuate  them  from  the  scene.  In  addition  to  going 

24Location  of  Baltimore  hospitals.  Retrieved  on  30  March  2006  from  the  World  Wide  Web  at 
http://www.  local.  com/results.aspx?kevword=hospital&location=Baltimore%2c+MD&radius=5 
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to  the  wounded  and  treating  them  in  place,  the  medical  teams  have  the 
responsibility  of  setting  up  triage  points  in  the  vicinity  of  the  attack. 

Fire  response  in  this  simulation  comes  from  two  nearby  fire  stations. 
Station  23  is  a  large,  fully  capable  station  located  approximately  three-quarters  of 
a  mile  west  of  the  scene  of  the  explosion.  Station  33  is  a  smaller  station  that  is 
about  three-quarters  of  a  mile  south  of  the  explosion.25  The  primary  mission  of 
the  fire  department  is  to  eliminate  the  threat  of  fire.  If  there  is  no  threat  of  fire,  the 
fire  department  will  assist  the  police  department  with  the  evacuation  of  civilians 
from  the  area  affected  by  the  bomb  blast.  The  fire  department  will  also  assist 
personnel  from  the  area  hospitals  in  setting  up  triage  points.  Stations  23  and  33 
respond  to  the  scene  with  similar  equipment:  ladder  trucks,  pumping  trucks, 
and  medics. 

After  the  initial  response  and  follow-on  response  by  the  first  alarm 
responders,  the  incident  commander  makes  a  full  assessment  and  will  request 
second  alarm  responders.  The  mix  of  police,  fire,  and  medical  responders  is 
dependent  on  the  situation  and  location.  This  simulation  does  not  address 
second  alarm  responders;  simulating  decisions  made  by  the  incident  command 
post  is  beyond  the  scope  of  this  research. 

D.  DESCRIPTION  OF  CIVILIANS 

Ten  thousand  civilians  are  in  the  area  of  the  Celebrate  Baltimore  festival 
when  the  terrorists  detonate  the  bomb.  There  are  three  different  classifications 
of  civilians  in  the  model:  those  unaffected  by  the  bomb,  those  who  are 
ambulatory  wounded,  and  those  that  are  stretcher  wounded.  There  are  four 
groups  of  the  unaffected  and  ambulatory  wounded  civilians  in  the  model:  men, 
women,  boys,  and  girls.  Each  group  has  separate  characteristics  in  the  model. 
All  stretcher  wounded  (men,  women,  boys,  and  girls)  have  the  same 
characteristics  in  the  model. 


25l_ocation  of  Baltimore  fire  stations.  Retrieved  on  30  March  2006  from  the  World  Wide  Web 
at  http://www.local.com/results. aspx?kevword=hospital&location=Baltimore%2c+MD&radius=5 
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1 .  Behavior  of  Unaffected  and  Ambulatory  Wounded  Civilians 

Unaffected  civilians  are  aware  of  the  bomb’s  detonation;  they  want  to 
leave  the  area.  Civilians  that  are  ambulatory  wounded  by  the  bomb  behave  in  an 
unpredictable  manner;  that  is,  they  panic.  When  civilians  panic,  they  will  respond 
to  attempts  by  police,  fire,  and  medical  personnel  to  calm  them.  Panicking 
civilians  will  also  respond  to  terrorist  attempts  to  instill  more  panic.  For  example, 
if  a  civilian  is  moving  toward  a  triage  point  because  a  policeman  gave  him 
directions,  a  terrorist  can  still  deter  the  civilian  and  cause  more  panic. 

2.  Stretcher  Wounded  Civilians 

Stretcher  wounded  civilians  must  be  stabilized  by  hospital  personnel 
(doctor,  nurse,  EMT)  before  they  can  move  to  a  triage  point. 

E.  DESCRIPTION  OF  TERRORISTS 

Five  terrorists  orchestrate  the  simulated  attack  on  the  Celebrate  Baltimore 
festival.  Four  terrorists  have  the  purpose  of  increasing  the  level  of  panic 
experienced  by  the  civilians  in  affected  area.  These  four  terrorists  are  in  civilian 
clothes  and  attempt  to  blend  into  the  crowd  so  as  to  not  be  readily  identified  by 
responding  police  officers.  They  do  not  have  a  weapon,  but  are  armed  with  the 
knowledge  of  how  to  incite  panic  in  a  crowd.  These  terrorists  focus  on  the  areas 
affected  by  the  explosion  that  have  the  largest  civilian  density. 

The  fifth  terrorist  is  a  sniper  that  will  directly  attack  responding  agencies  to 
disrupt  their  influence  on  the  crowd.  The  sniper  is  armed  with  an  AK-47  assault 
rifle  and  is  wearing  body  armor.  This  sniper  hides  in  a  building  during  the  initial 
response  and  first  alarm  response.  After  the  first  alarm  responders  have  arrived 
on  scene,  the  sniper  will  begin  to  shoot  police,  fire,  and  medical  targets  of 
opportunity.  The  goal  of  the  sniper  is  to  slow  the  emergency  response  and  incite 
panic  in  the  crowd  by  slowly  attriting  response  forces. 

Now,  with  an  understanding  of  the  scenario  and  an  overview  of  the  forces 
simulated,  a  thorough  discussion  of  the  model  follows.  In  addition,  a  detailed 
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description  of  methods  used  to  replicate  behaviors  and  characteristics  of  various 
weapon  systems  is  included. 
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III.  MODELING  METHODOLOGY 


This  chapter  provides  a  general  background  of  simulation  and  MAS,  and 
specific  details  of  the  methodology  developed  to  model  first  response  to  a  crisis 
situation  in  a  MAS  environment.  It  begins  with  a  brief  overview  of  simulation 
models  and  leads  to  a  summary  of  multi-agent  simulations  and  their  place  in 
combat  modeling.  The  author  establishes  the  reasons  for  using  MAS  to  analyze 
the  chosen  situation  and  the  selection  of  Pythagoras  as  the  particular  model 
used.  This  section  concludes  with  a  detailed,  step-by-step  discussion  of  a 
methodology  that  applies  MAS  techniques  to  modeling  first  response  in  a  crisis 
situation,  and  a  discussion  of  the  methodology’s  application. 

A.  SIMULATION  SUPPORT  TO  MILITARY  ANALYSIS 

The  Department  of  Defense  (DoD)  relies  on  simulation  models  to  capture 
significant  insights  that  enable  senior  leadership  to  make  informed  decisions. 
Insights  gained  from  simulation  models  become  especially  important  in  areas  in 
which  experimentation  is  impractical  or  impossible.  For  example,  it  is  nearly 
impossible  to  conduct  actual  physical  experiments  to  determine  the  effectiveness 
of  strategic  or  operational  war  plans,  large-scale  fielding  of  new  force  designs,  or 
weapon  system  capability  changes  in  actual  conflict.26 

Traditional  combat  simulations  are  often  complex,  [high  resolution] 
simulations  that  require  a  large  amount  [of]  input  data,  an 
experienced  and  [well-trained]  staff  to  operate  and  months  to 
prepare  specific  scenarios  for  use.  These  issues,  as  well  as  slow 
runtimes,  make  traditional  simulations  both  time  and  resource 
intensive.  The  time  and  resource  requirements  often  force  an 
analyst  to  conduct  a  very  limited  set  of  simulation  runs  thereby  only 
focusing  on  a  limited  region  of  a  [model’s]  input  space.27 


26LTC  Thomas  M.  Cioppa,  “A  Potential  Role  of  Agent  Based  Models  in  Military  Analysis,” 
Information  Paper,  (prepared  December  2003). 

27lbid. 
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An  example  of  why  high  resolution  models  have  slow  run  times  extends 

from  simply  capturing  the  effects  of  firing  a  round  from  a  howitzer. 

To  replicate  a  howitzer  firing  a  projectile  in  a  high  resolution  model, 
the  analyst  must  know  more  information  [than]  just  the  classical 
‘trajectory  in  a  vacuum’  physics  problem.  Instead,  the  analyst  must 
take  into  account  interior,  exterior,  and  terminal  ballistics.  Each 
includes,  but  is  not  limited  to,  factors  such  as  projectile  square 
weight,  propellant  temperature,  propellant  moisture,  muzzle  velocity 
variation,  and  tube  wear  effecting  interior  ballistics,  as  well  as 
meteorological  atmospheric  conditions  such  as  air  temperature,  air 
moisture,  wind  direction,  wind  speed,  and  the  rotation  of  the  [earth] 
[affecting]  exterior  ballistics.  These  examples  only  name  a  few 
factors  that  the  analyst  could  consider  when  modeling  the  howitzer 
firing  the  projectile.  This  process  then  repeats  for  every  other 
howitzer  in  the  battery,  positioned  at  different  locations,  and  any 
other  munitions  also  fired.  As  such,  a  simulation  requiring  multiple 
munitions,  from  several  platforms  demands  significant  computing 
ability  just  to  provide  the  decision  maker  with  useful  insights 
required  for  his  decision.28 

The  above  is  an  example  of  the  reason  that  simultaneous  analysis  of  a 
wide  range  of  issues  is  difficult  with  high  resolution  models.  In  addition,  the 
traditional  deterministic  and  even  stochastic  techniques  that  are  used  in  these 
models  cannot  capture  key  elements  of  combat  uncertainty.  Factors  such  as 
synergy  among  actors,  command  and  control  influences,  and  human  adaptivity — 
long  considered  strengths  in  the  American  military — are  not  well  accounted  for  in 
traditional,  high  resolution  models.  The  shortcomings  of  traditional  models  led  to 
the  search  for  a  new  tool.29 

An  innovative  class  of  low  resolution  simulation,  known  as  MAS,  emerged 
to  complement  and  augment  previously  established,  more  computationally 
intensive,  physics-based,  simulation  models.  The  role  of  MAS  is  not  to  replace 


28Charles  A.  Sulewski,  “An  Exploration  of  Unmanned  Aerial  Vehicles  in  the  Army’s  Future 
Combat  Systems  Family  of  Systems,”  Masters  Thesis,  Naval  Postgraduate  School,  Monterey, 
CA,  December  2005. 

29Raymond  R.  Hill,  et  al.,  “Some  Experiments  With  Agent-Based  Combat  Models,”  Military 
Operations  Research,  Vol.  8,  No.  3,  2003,  pp.  17-28. 
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high  resolution  models,  however,  MAS  increasingly  proves  useful  to  the  DoD  in 
two  primary  areas. 

MAS  may  be  used  in  exploratory  analysis,  to  gain  quick  insight  and 
narrow  the  focus  of  many  factors,  parameters,  and  variables  to  expedite  using 
high  resolution,  physics-based  simulations.30  Narrowing  the  field  of  factors  to 
explore  saves  time  and  money  at  the  beginning  of  a  simulation  study.  MAS  is 
also  used  to  address  key  objectives  that  would  otherwise  require  extensive  use 
of  resource-intensive,  physics-based  models.  The  analyst  may  switch  back  and 
forth  between  two  models  to  gain  advanced  scenario  insight.31 

Analysis  of  a  MAS  leads  to  a  relatively  fast,  thorough  exploration  of  the 
solution  space  associated  with  the  given  problem.  MAS  offers  quick  scenario 
generation,  fast  run  times,  rapid  data  turn  around,  and  permits  the  analyst  to 
consider  many  alternatives  in  a  short  amount  of  time.  MAS  complements  and 
augments  physics-based  models,  permitting  analysts  to  examine  the  problem 
over  a  greater  range  of  plausible  possibilities. 

MAS  is  an  appropriate  method  to  help  an  analyst  provide  insights  that 
support  decision  makers  in  situations  involving  crisis  response.  Crisis  response 
involves  human  decision-making,  which  can  be  unpredictable,  especially  under 
stressful  situations.  Decisions  made  by  individuals  can  lead  to  global  patterns  or 
emergent  behavior.  MAS  can  help  identify  possible  patterns  and  behaviors. 
MAS  also  offers  the  analyst  the  ability  to  run  a  model  thousands  of  times  to  gain 
insight  into  hundreds  or  thousands  of  different  “thought  processes.”  In  many 
circumstances,  especially  at  the  beginning  of  analysis,  this  insight  is  more 
valuable  than  providing  a  handful  of  point  estimates  that  come  from  relatively 
long,  expensive,  high  resolution  runs. 


30LTC  Thomas  M.  Cioppa,  “A  Potential  Role  of  Agent  Based  Models  in  Military  Analysis,” 
Information  Paper,  (prepared  December  2003). 

31  Ibid. 
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B.  PYTHGORAS  BACKGROUND 


The  author  chose  Northrop  Grumman’s  Pythagoras  as  the  agent-based 
simulation  modeling  tool  to  support  this  research  because  its  developers  included 
several  features  in  this  model  that  are  especially  useful  in  modeling  crisis 
response.  Crisis  response  is  inherently  complex:  a  multisided  event  with  many 
interrelationships  and  a  requirement  to  gradually  escalate  force.  Pythagoras 
offers  a  set  of  capabilities  that  is  unique  in  the  area  of  agent-based  simulations:32 

•  Incorporates  soft  rules  to  distinguish  unique  agents 

•  Uses  desires  to  motivate  agents  into  moving  and  shooting 

•  Includes  the  concept  of  affiliation  (established  by  sidedness,  or 
color  value)  to  differentiate  agents  into  members  of  a  unit,  friendly 
agents,  neutrals,  or  enemies 

•  Allows  for  behavior-changing  events  and  actions  (called  triggers) 
that  may  be  invoked  in  response  to  simulation  activities 

•  Retains  traditional  weapons,  sensors,  and  terrain 

Northrop  Grumman  initially  developed  Pythagoras  in  support  of 
Project  Albert,  a  U.S.  Marine  Corps-sponsored  international  initiative  focusing  on 
human  factors  in  combat  and  noncombat  situations.  They  have  updated  the 
model  as  needed  and  as  requested,  including  several  updates  during  the 
development  of  this  thesis. 

The  developers  of  Pythagoras  built  this  model  with  the  intent  that  an 
individual  without  a  hard  science  degree  could  learn  to  effectively  use  the  model 
within  eight  hours.33  The  company’s  goal  of  simplicity  paid  off.  Pythagoras  offers 
a  simple-to-use  Graphical  User  Interface  (GUI),  including  drop-down  window 
capabilities  much  like  many  Windows-based  applications.  Within  a  few  short 
hours  at  the  console,  a  Pythagoras  novice  can  input  a  meaningful  scenario  and 
start  gaining  insights  into  the  problem  at  hand. 

32Henscheid,  Z.,  Pythagoras  User’s  Manual,  Version  1 .9,  February  2006,  p.  18.  Retrieved 
from  the  World  Wide  Web  on  5  February  2006  at  http://www.projectalbert.org/downloads.html. 

33lbid. 
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C.  METHODOLOGY  OVERVIEW 

The  main  objective  of  this  research  is  to  provide  a  methodology  by  which 
emergency  first  response  to  a  crisis  event  can  be  simulated  in  a  MAS 
environment.  This  methodology  expands  on  areas  two,  three,  and  four  of 
Figure  2.  Figure  6  provides  an  illustration  of  the  developed  methodology, 
followed  by  a  description  of  the  diagram.  Many  of  the  steps  are  interrelated, 
iterative,  and  flexible;  the  given  situation  may  require  some  of  the  steps  to  be 
accomplished  in  a  different  order  than  outlined  in  Figure  6. 
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The  focus  of  this  section  is  on  steps  1  through  7  of  the  methodology.  The 
author  explains  steps  8  and  9  in  detail  in  Chapter  IV  (Model  Implementation). 
Chapter  V  (Design  Methodology)  focuses  on  steps  10  and  11.  Chapters  VI 
(Data  Analysis)  and  VII  (Conclusions  and  Recommendations  for 
Future  Study)  detail  the  procedures  for  steps  12  and  13,  respectively. 

The  vignette  modeled  for  this  thesis  is  a  terrorist  attack  in  a  large  city 
(Baltimore)  during  a  festival.  This  simulation  is  an  application  of  the 
methodology,  but  this  methodology  is  applicable  to  response  in  many  other 
crises:  manmade  and  natural,  large  scale  and  small. 

1.  Identify  Event  Type  and  Location 

The  first  step  in  addressing  a  problem  in  a  scientific  manner  is  to  define 
the  problem.  Simulating  emergency  first  response  begins  with  knowing  the  event 
and  the  location.  Response  to  a  terrorist  attack  is  different  than  response  to  a 
natural  disaster.  Though  the  resources  and  priorities  of  work  differ  between  a 
natural  and  a  man-made  attack,  the  methodology  used  for  simulating  them  is 
the  same. 

2.  Identify  Level  of  Governmental  Response 

The  analyst  must  consider  several  factors  when  deciding  on  the 
appropriate  level  of  governmental  response  to  analyze.  Identifying  the 
requirements  in  SOPs  that  govern  crisis  response  in  the  appropriate  situation  is 
the  first  step,  since  procedures  in  different  crisis  events  call  for  different  levels  of 
response.  For  example,  federal  agencies  will  respond  to  a  terrorist  attack  without 
the  requirement  of  being  requested  by  the  local  government.  However,  in  a 
natural  disaster,  the  local  government  may  need  to  request  federal  assistance. 

In  addition  to  verifying  the  procedurally  appropriate  level  of  response,  the 
analyst  must  consider  the  goals  of  the  decision  maker.  In  a  situation  that  would 
usually  involve  the  response  of  the  federal  government,  the  decision  makers 
could  be  city  planners  who  are  seeking  to  validate  local  plans  without 
federal  assistance. 
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3.  Identify  Level  of  First  Response 

Identifying  which  response  agencies  to  include  is  another  important  step  in 
scoping  the  problem.  Again,  the  modeler  must  identify  what  the  decision  maker 
deems  is  important.  If  the  simulation  is  meant  to  support  a  police  chiefs  decision 
concerning  force  allocation,  inclusion  of  other  agencies  may  not  be  a 
requirement.  In  a  simulation  meant  to  support  federal  emergency  response,  it  is 
important  to  simulate  all  facets  of  emergency  response. 

4.  Establish  Interagency  Communication 

If  all  agencies  being  simulated  belong  to  the  same  organization,  lines  of 
communication  for  information  sharing  are  most  likely  in  place.  Some 
organizations  may  have  different  parents;  the  next  step  in  this  simulation 
methodology  is  to  establish  communication  between  like  offices  in  the  different 
organizations.  Information  sharing  is  the  key  to  a  realistic,  credible  simulation  of 
crisis  response. 

5.  Enumerate  Focus  Areas  and  Questions  to  be  Answered 

With  the  background  for  the  simulation  set  and  the  lines  of  communication 
open,  the  focus  shifts  to  the  goals  the  decision  makers  are  seeking  to  achieve. 
The  analyst  must  identify  areas  of  interest  in  which  the  decision  maker  wants  to 
gain  insights.  Narrowly  scoped  research  questions  will  guide  the  direction  of  the 
simulation  later  in  development.  With  the  areas  of  interest  and  research 
questions  developed,  the  analyst  can  review  and  prioritize  agency  SOPs  for 
priorities  of  work  needed  to  address  the  focus  areas  and  research  questions. 

6.  Identify  Priorities  of  Work  Relevant  to  Focus  Areas 

Identify  priorities  of  work  that  occur  automatically  after  the  crisis  event; 
also,  identify  priorities  of  work  that  require  decisions  to  be  made  after  the  event 
occurs.  The  simulation  can  easily  include  decisions  that  are  made  before  the 
event.  However,  MAS  are  simulations  that  are  “human-out-of-the-loop.”  Actual 
human  decisions  stop  when  a  constructive  MAS  starts.  When  the  modeler 
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encounters  a  place  in  the  simulation  where  an  actual  human  decision  should  be 
made,  i.e.,  a  decision  beyond  the  scope  of  the  simulation’s  agents,  there  are  two 
ways  to  proceed.  First,  the  modeler  can  end  the  simulation  at  the  point  of  the 
human  decision.  Alternatively,  if  the  decision  maker  needs  the  simulation  to  run 
longer,  the  modeler  can  make  the  decision  before  the  model  is  run.  Each 
decision  is  fully  documented,  complete  with  the  reasons  for  making  the  decision. 

During  the  process  of  establishing  priorities  of  work,  the  analyst  begins  to 
develop  the  list  of  factors  included  in  the  model  that  are  most  important  to  the 
decision  maker.  During  the  experimental  design  stage  of  this  methodology,  the 
analyst  will  vary  these  factors  to  conduct  the  analysis  that  will  answer  the 
questions  posed  at  the  beginning  of  the  project. 

7.  Choose  the  Model 

When  priorities  of  work  are  established  and  prioritized,  the  analyst  must 
recognize  the  actions  taken  to  accomplish  those  priorities.  The  simulation  must 
reflect  reality.  The  actions  taken  to  accomplish  priorities  of  work  are  important 
because  it  is  these  actions  that  determine  which  simulation  model  to  use. 

It  may  be  better  to  use  Pythagoras  to  simulate  a  model  in  which  the 
decision  maker  is  interested  in  leadership  qualities,  behavior  tolerances,  or 
affiliation  changes.  The  decision  to  use  Pythagoras  to  model  the  TOPOFF 
scenario  is  discussed  in  Section  III.B.  Alternatively,  it  may  be  most  appropriate 
to  use  Map  Aware  Non-uniform  Automata  (MANA)  if  the  decision  maker  is 
interested  in  communication  architecture  or  situational  awareness.34  The 
decision  maker  may  choose  NetLogo  if  interested  in  the  dynamics  of  crowd 
control  or  mob  effects.35 


34David  P.  Galligan,  Mark  A.  Anderson,  and  Michael  K.  Lauren,  Map  Aware  Non-uniform 
Automata,  Version  3.0,  July  2004. 

35Uri  Wilensky,  NetLogo  User’s  Manual,  Version  3.1,  April  2006.  Retrieved  from  the 
World  Wide  Web  on  1 9  May  2006  at  http://ccl.sesp.northwestern.edu/netloqo/docs 
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8.  Create  Small  Vignettes 

After  choosing  the  model,  begin  modeling  small  portions  of  the  overall 
scenario.  Start  by  modeling  and  debugging  similar  agencies  or  priorities,  or  work 
separately  to  simplify  interactions  among  agents.  See  Chapter  IV 
(Model  Implementation)  for  detailed  information  on  how  to  create  vignettes  in  a 
MAS,  specifically,  Pythagoras. 

9.  Merge  Separate  Agency  Vignettes  and  Debug 

After  debugging  separate  vignettes,  merge  the  separate  vignettes  into  one 
large  scenario.  Add  and  debug  one  vignette  at  a  time  to  minimize  time  spent  in 
locating  errors  in  the  model. 

10.  Calibrate  the  Model 

Model  calibration  gains  the  developed  simulation  a  measure  of  credibility. 
The  model  developer  compares  the  outputs  of  the  simulation  to  other,  previously 
established  outputs.  For  example,  an  analyst  may  compare  the  outputs  of  the 
model  to  events  that  actually  happened,  if  applicable.  Another  way  to  calibrate  a 
model  is  to  statistically  compare  the  model’s  outputs  to  that  of  another  model, 
possibly  a  validated  or  accredited  model. 

1 1 .  List  Factors  of  Interest 

Developing  factors  of  interest  is  an  ongoing  process  that  begins  when  the 
decision  maker’s  priorities  are  established.  Most  relevant  work  on  this  step 
occurs  during  the  time  that  the  scenario  developer  is  creating,  debugging,  and 
merging  vignettes.  After  the  vignettes  are  merged  into  one  scenario  and 
debugged,  the  factor  list  can  be  formalized,  complete  with  the  corresponding 
levels  that  provide  the  decision  maker  with  desired  insights.  The  analyst  must 
clearly  identify  the  desired  effects  of  varying  a  given  parameter.  Chapter  V 
(Design  Methodology)  provides  detailed  information  on  how  to  accomplish  steps 
10  and  11. 


31 


12.  Varying  Factors  of  Interest  in  Experimental  Design 

After  the  simulation  is  complete  and  debugged,  use  the  power  of  modern 
computers  and  supercomputers  to  “data  farm.” 

Data  farming  is  the  process  of  using  a 
[high-performance]  computer  or  computing  grid  to  run 
a  simulation  thousands  or  millions  of  times  across  a 
large  parameter  and  value  space.  The  result  of 
Data  Farming  is  a  ‘landscape’  of  output  that  can  be 
analyzed  for  trends,  anomalies,  and  insights  in 
multiple  parameter  dimensions.36 

13.  Analyze  Data 

It  is  through  the  analysis  of  the  data  “grown”  by  the  use  of  an  efficient 
experimental  design  and  data  farming  that  insights  and  lessons  learned  are 
obtained  and  documented,  and  then  provided  to  the  decision  maker.  See 
Chapter  VI  (Data  Analysis)  for  information  about  applicable  data  analysis 
techniques  and  results  from  the  Baltimore  Attack  scenario. 

14.  Document  Insights  and  Lessons  Learned 

Following  the  analysis  of  the  model  and  factors,  it  is  possible  to  refine  the 
model  and  repeat  the  experiment.  If  the  model,  experimental  design,  and 
resultant  data  analysis  meet  the  decision  maker’s  needs,  the  analyst  must 
document  findings  and  lessons  learned.  See  Chapter  VII  (Conclusions  and 
Recommendations  for  Future  Study)  for  documentation  of  the  findings  and 
recommendations  resultant  from  analyzing  the  Baltimore  Attack  scenario. 

D.  METHODOLOGY  APPLICATION 

This  section  demonstrates  how  the  previously  described  methodology 
applies  to  simulating  the  TOPOFF  3  scenario.  Each  step  is  enumerated, 
followed  by  a  concise  bullet  comment  that  summarizes  how  the  step  is  achieved 
in  this  research. 


36Definition  of  Data  Farming.  Retrieved  on  9  May  2006  from  the  World  Wide  Web  at 
http://en.wikipedia.org/wiki/Data  farming 
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1.  Identify  Event  Type  and  Location 

•  TOPOFF  3  scenario,  large  bomb  VBIED,  New  London,  CT 

•  Move  scenario  to  Baltimore  with  same  characteristics 
as  above 

2.  Identify  Level  of  Governmental  Response 

•  Focus  on  city  planning,  do  not  use  federal  and/or 
state  assets 

3.  Identify  Level  of  First  Response 

•  Police,  fire,  and  medical  response 

4.  Establish  Interagency  Communication 

•  Not  applicable  in  this  simulation,  proof  of  concept  exercise 

5.  Enumerate  Focus  Areas  and  Questions  to  be  Answered 

•  Communication:  What  is  the  most  effective  interagency 
communication  architecture  for  emergency  response  to 
VBIED  in  Baltimore’s  Inner  Harbor? 

•  Determination  of  Assets  Required:  What  is  the  most 
appropriate  mix  of  police,  fire,  and  medical  assets? 

6.  Identify  Priorities  of  Work  Relevant  to  Focus  Areas 

•  Identify/neutralize  additional  threats 

•  Coordinate  initial  and  follow-on  response 

•  Identify  dead  and  wounded 

•  Stabilize  wounded,  move  to  triage  site 

•  Restore  calm 

7.  Choose  the  Model 

•  Use  Pythagoras 

8.  Create  Small  Vignettes 

•  Terrorist  agitators  incite  panic  in  crowd 


33 


•  Police  on  patrol,  looking  for  wounded,  threats 

•  Arrival  of  medics,  treatment  of  stretcher  wounded 

•  Incident  command  post,  fusion  of  communication  across 
different  agencies 

•  Integration  of  SWAT  team,  response  to  threat 

•  Arrival  and  deployment  of  fire  department 

•  Arrival  and  deployment  of  police  follow-on  response 

•  Terrorist  gunman  hides  and  attacks  targets  of  opportunity 

9.  Merge  Separate  Agency  Vignettes  and  Debug 

•  The  small  vignettes  are  merged  into  one  overall  simulation, 
see  Chapter  IV  (Model  Implementation) 

10.  Calibrate  the  Model 

•  A  rigorous  way  to  calibrate  this  simulation  model  is  to 
generate  a  distribution  of  outputs,  then  compare  the  results 
to  one  EPiCS  run,  or  a  distribution  of  outputs  gained  from 
several  EPiCS  runs.  The  capability  does  not  currently  exist 
to  run  several  EPiCS  simulations  and  capture  the 
needed  data. 

•  The  author  calibrated  this  model  through  the  use  of  modeling 
subject  matter  expertise  at  the  Naval  Postgraduate  School 
and  his  own  personal  expertise  in  the  area  of  emergency 
first  response  operations.  This  calibration  was  conducted 
using  a  qualitative  comparison  of  the  outputs  this  simulation 
and  EPiCS  can  provide. 

1 1 .  List  Factors  of  Interest 

•  See  Chapter  V  (Design  Methodology) 

12.  Varying  Factors  of  Interest  in  Experimental  Design 

•  See  Chapter  V  (Design  Methodology) 

13.  Analyze  Data 

•  See  Chapter  VI  (Data  Analysis) 
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14.  Document  Insights  and  Lessons  Learned 

•  See  Chapter  VII  (Conclusions  and  Recommendations  for 
Future  Study) 

This  section  has  provided  the  reader  with  a  general  overview  of 
simulations  and  specific  information  about  the  MAS,  Pythagoras.  The  author 
establishes  and  explains  a  methodology  by  which  an  analyst  can  use  MAS  to 
model  emergency  first  response  to  a  crisis  event.  The  section  concludes  with  an 
overview  of  how  Pythagoras  can  be  used  in  the  application  of  the  methodology  to 
model  a  TOPOFF  scenario.  In  subsequent  sections,  the  “nuts  and  bolts”  of  the 
model  are  explained,  including  settings,  assumptions,  and  limitations.  In 
addition,  the  process  of  experimental  design  in  this  simulation  is  discussed,  with 
the  resultant  data  analysis  and  conclusions. 
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IV.  MODEL  IMPLEMENTATION 


This  chapter  provides  a  detailed,  step-by-step  discussion  of  the 
techniques  used  to  model  emergency  response  to  a  crisis  in  a  MAS.  The 
modeling  described  here  corresponds  to  steps  8  and  9  of  the  methodology 
outlined  in  Figure  6  and  subsequently  discussed.  Additionally,  this  chapter  will 
touch  on  step  10,  calibration  of  the  model.  The  following  includes  details  that 
address  Pythagoras’s  model  settings  specifically,  but  can  be  generalized  to  be 
applicable  to  other  models.  The  chapter  concludes  with  a  discussion  of  known 
“bugs”  in  the  model  and  the  model’s  limitations. 

A.  MODELING  TECHNIQUE 

This  section  contains  general  information  that  can  be  used  to  create  the 
scenario  within  most  any  MAS.  In  addition,  this  section  contains  detailed 
information  that  describes  how  this  simulation  was  created  in  Pythagoras. 
Appendix  A  (Model  Implementation)  contains  Excel  tables  and  Pythagoras 
screen  shots  that  illustrate  ideas  discussed  here  in  writing.  The  first  section  of 
Appendix  A  outlines  the  creation  of  the  environment  in  which  the  simulation 
occurs.  It  is  the  translation  of  the  actual  world  into  the  simulation  world.  In  the 
subsequent  sections,  the  author  describes  the  different  types  of  agents  in  the 
scenario  and  the  interactions  among  agents  in  each  major  event.  The  detailed 
discussion  includes  the  goals  of  each  organization  involved,  modeling  techniques 
used  to  capture  the  effects  of  organizational  operations,  and  the 
assumptions  made. 

The  author  does  not  claim  that  the  tactics  modeled  in  this  simulation  are 
optimal  emergency  response  tactics,  techniques,  and  procedures  (TTPs).  This 
simulation  does  provide  a  proof  of  concept  that  it  is  possible  to  model  emergency 
response  priorities  and  TTPs  in  a  multi-agent  environment.  The  power  of  this 
tool  is  that  it  is  possible  to  quickly  adapt  this  model  and  this  scenario  to  analyze 
many  different  force  structures,  TTPs,  organizational  priorities,  and  even  other 
crisis  events. 
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1. 


Overall  Model  Configuration 


a.  Scaling  the  Model 


The  first  step  of  using  a  simulation  model  is  scaling  the  model’s 
workspace  to  represent  the  area  of  interest.  Traditional  units  of  measure  include 
feet,  miles,  meters,  or  kilometers.  This  scenario  includes  a  “terrain  box”  that  is 
four  million  square  feet  of  the  Baltimore  Inner  Harbor,  2,000  feet  long  by 
2,000  feet  wide  (see  Figure  7).  The  unit  of  measure  in  Pythagoras  is  the  pixel; 
the  size  of  the  model’s  terrain  is  1,000  pixels  by  1,000  pixels.  In  this  vignette, 
one  pixel  represents  two  feet.  Appendix  A  includes  a  detailed  discussion  about 
the  scaling  of  this  simulation,  including  the  measurement  of  distances. 


Figure  7.  Comparison  of  Actual  Terrain37  in  Baltimore’s  Inner  Harbor  to  Terrain  in  Model 


37Graphic  retrieved  from  Google  Earth,  12  April  2004;  locations  are  adapted  from 
TRAC-WSMR  EPiCS  simulation  conducted  in  February  2006.  Free  application  downloaded  on 
30  March  2006  from  the  World  Wide  Web  at  http://earth.qooqle.com/download-earth.htm. 
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b.  Timestep 

Time  in  Pythagoras  is  measured  by  timesteps.  It  is  important  to 
give  careful  consideration  to  the  determination  of  the  translation  from  seconds  to 
timesteps.  Agents  move  from  their  previous  position  to  their  subsequent  position 
by  jumping  directly  to  the  desired  position.  Agents  do  not  pass  through  the  pixels 
between  positions.  A  timestep  that  represents  a  long  period  of  real-time  could 
result  in  agents  “jumping”  past  each  other  and  not  interacting.  A  timestep  that 
represents  too  little  real  time  could  result  in  the  simulation  requiring  a  vast 
amount  of  CPU  time  to  run.  For  this  vignette,  the  author  chose  one  timestep  to 
represent  four  seconds,  resulting  in  a  simulation  of  one  hour  of  time  that  runs  for 
900  timesteps. 


c.  Terrain 

Terrain  is  defined  by  its  characteristics.  A  given  terrain  feature  will 
have  a  certain  height;  it  may  offer  visibility  through  it,  restrict  movement,  and 
have  a  protective  value.  Different  types  of  terrain,  as  seen  in  Figure  6,  have 
different  colors,  which  are  chosen  by  the  user,  not  the  developers.  Gray  terrain 
in  one  simulation  may  be  easily  trafficable  concrete;  in  another  simulation,  gray 
could  be  an  impenetrable  wall.  In  Pythagoras,  concealment  occurs  in  three 
“bands” — A,  B,  and  C.  These  bands  are  associated  with  the  possible  selections 
for  available  sensors  in  the  model.  More  discussion  about  sensing  bands  follows 
in  Section  IV.A.I.f.  See  Table  1  for  Pythagoras  terms  associated  with  its  terrain. 
Specific  characteristics  for  all  weapons  included  in  this  simulation  are  located  in 
Appendix  A,  Section  A.1  .c. 


Subject 

Actual 

Terminology 

Pythagoras 

Language 

Scale 

Height 

Height  (feet) 

Height  (pixels) 

0  -  1 ,000 

Sight 

Visibility 

Concealment 

0.0-  1.0 

Movement 

Trafficability 

Movement  Factor 

0.0-  1.0 

Protection 

Cover 

Protection 

0.0-  1.0 

Table  1.  Translation  of  Pythagoras  Terrain  Terms 
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Concealment,  movement  factor,  and  protection  are  multipliers  that 
affect  the  probability  of  detection,  rate  of  movement,  and  probability  of  being 
killed.  For  example,  a  wall  made  of  cinder  blocks  may  have  a  height  of  20  feet 
(10  pixels).  A  person  cannot  see  through  the  wall,  so  the  concealment  factor  is 
1 .0.  The  probability  of  detection  results  from  the  following  formula: 

Actual  PD  =[1- (Concealment  Factor)]  *  Baseline  PD 

A  person  cannot  walk  through  a  wall  that  is  20  feet  tall,  thus  the 
movement  factor  would  be  0.0;  therefore,  the  movement  rate  is  0:  0.0  *  standard 
movement  rate. 

Suppose  the  weapons  included  in  the  simulation  cannot  shoot 
through  a  cinder  block  wall.  The  protection  value  would  be  1 .0.  The  probability 
of  kill  for  a  weapon  trying  to  shoot  through  the  wall  is  0,  because  Pythagoras 
includes  (1 -protection  factor)  as  a  multiplier  in  determining  actual  probability  of 
kill  for  a  given  situation.  The  author  addresses  movement  rates  in  greater  detail 
in  Section  IV.A.3.a.  See  Table  2  for  the  types  of  terrain  included  in 
this  simulation. 


Color 

Name 

Height 

Concealment 

Movement 

Factor 

Protection 

iGray 

Concrete 

0.0 

0.0 

1.0 

0.0 

Black 

Road 

0.0 

0.0 

1.0 

0.0 

|  Light  Green 

Water 

0.0 

0.0 

0.0 

0.0 

Purple 

Building 

28 

1.0 

0.0 

1.0 

Dark  Purple 

Tall  Bldg 

60 

1.0 

0.0 

1.0 

Tan 

Inside  Bldg 

0.0 

0.0 

1.0 

0.0 

Light  Purple 

Rubbling 

3  pixels 

0.5 

0.2 

0.0 

Red 

Triage  Site 

0.0 

0.0 

1.0 

0.0 

Table  2.  Terrain  Colors  Used  in  Baltimore  Attack 
d.  Weapons 

One  way  that  an  agent  interacts  with  other  agents  is  through  the 


use  of  “weapons.”  Weapons  in  most  simulation  models,  and  Pythagoras  in 
particular,  are  not  limited  to  killing  enemies.  Table  3  describes  the  different  types 
of  influence  that  Pythagoras  weapons  can  impart.  In  general,  weapons  influence 
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the  target’s  health,  sidedness,  or  another  user-selected  attribute.  Changes  in 
sidedness  and  attributes  are  discussed  in  detail  later  in  this  section. 


Weapon  Class 

Weapon  Effect 

Example  Usage 

Killing 

Decrease  Target’s  Health 

Policeman’s  Pistol 

Restorative 

Increase  Target’s  Health 

EMT’s  Medical  Kit 

Paintball 

Incremental  Side  or  Attribute  Change 

Propaganda  Leaflets 

Table  3.  Pythagoras’s  Weapon  Effects 


Table  4  provides  a  breakdown  of  the  different  weapons  used  in  this 
simulation;  who  uses  them  and  their  effects.  Appendix  A  includes  detailed 
weapon  information.  The  United  States  Army  Materiel  Systems  Analysis  Activity 
(AMSAA)  provided  weapon  characteristics  for  the  M4  rifle,  M9  pistol,  and  AK-47 
assault  rifle.  Characteristics  of  other  weapons  are  the  result  of  the  author’s 
experience  as  a  military  policeman,  in  addition  to  informal  conversations  with 
police,  Special  Forces  operators,  and  other  subject  matter  experts.  More 
detailed  information  about  agent  side  properties  is  included  in  Appendix  A, 
Section  A.I.d. 


Weapon 

WPN 

Class 

Weapon  Carrier 

Target 

Effect 

9mm 

K* 

Police 

Enemies 

Decrease  Health 

AK-47 

K 

Terrorist  Gunman 

Enemies 

Decrease  Health 

Agitator 

p** 

Covert  Terrorist 

Enemies 

Cause  Panic 

Bomb 

K 

None 

All 

Decrease  Health,  Cause  Panic 

Rock 

K,  P 

Panicking  Civilians 

All 

Decrease  Health,  Cause  Panic 

M4 

K 

SWAT 

Enemies 

Decrease  Health 

Medical  Kit 

p*** 

EMT 

Friends 

Increase  Health 

Orders 

P 

Police,  EMTs,  Firemen 

Friends 

Calm  Panic,  Send  to  Med  Station 

*K  denotes  “killing”  weapon,  lethal  effects 

**P  denotes  “paintball”  weapon,  nonlethal  effects 

***R  denotes  “restorative”  weapon,  healing  effects 


Table  4.  Weapons  Modeled  in  Baltimore  Scenario 

The  effect  that  a  given  weapon  has  on  its  target  is  calculated  by 
multiplying  the  following  factors: 

•  Probability  of  hit  associated  with  the  weapon,  as  a  function  of  range 

•  Shooting  agent’s  marksmanship 

•  Probability  of  kill  of  the  weapon  (given  a  hit) 


41 


The  effectiveness  of  the  weapon  (lethality) 
Vulnerability  of  the  agent  being  shot 
Protection  factor  offered  by  the  terrain 


e.  Agent  Side  Property 

Side  property  is  a  feature  that  is  unique  to  the  Pythagoras  model, 
allowing  the  inclusion  of  many  different  sides  in  a  simulation,  not  just  friends  and 
enemies.  The  modeler  assigns  sides  to  the  agents  using  color  combinations  of 
blue,  green,  and  red.  Each  color  is  established  on  an  integer  scale  between  0 
and  255.  The  modeler  then  establishes  a  color  radius  to  determine  unit 
membership,  friendship,  neutrality,  and  who  is  an  enemy.  Agents  will  assess 
each  other’s  color  and  compare  it  to  their  own  color  to  determine  affiliation, 
based  on  the  distance  between  the  two  agents’  colors  and  the  unit,  friend,  and 
enemy  color  radii. 

Table  5  shows  a  simple  example  of  friend,  neutral,  and  enemy 
affiliation,  using  only  the  color  blue. 


Agent  1 

Unit 

Radius 

Friend 

Radius 

Enemy 

Radius 

Agent  2 

Result 

255,  0,  0 

10 

20 

50 

255,  0,  0 

Unit  Member 

255,  0,  0 

10 

20 

50 

240,  0,  0 

Friend 

255,  0,  0 

10 

20 

50 

210,  0,  0 

Neutral 

255,  0,  0 

10 

20 

50 

150,  0,  0 

Enemy 

Table  5.  Determination  of  Affiliation  by  Color 

Pythagoras  enables  the  modeler  to  use  any  combination  of  blue, 
green,  and  red  to  determine  affiliation.  In  using  combinations  of  two  or  more 
colors  to  determine  affiliation,  the  modeler  must  decide  on  one  of  two  methods  to 
calculate  distance  between  the  agents’  colors. 

Manhattan  Distance  (used  in  this  thesis)  is  the  result  of  adding  the 
color  differences  in  each  of  the  three  colors,  and  is  calculated  by  using  the 
following  formula. 

d  =  |Blue2  -Blue1|  +  |Green2  -Green1|  +  |Red2  -Red^ 
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The  mathematical,  or  straight  line,  distance  between  the  two  colors, 
also  called  the  Euclidean  Distance,  is  calculated  using  the  following  formula. 

d  =  y[(  Blue2  -Blue.,)2  +(Green2  -Green.,)2  +(Red2  -Red.,)2 

There  are  many  sides  represented  in  this  simulation.  There  are  the 
public  servants  (police,  fire,  and  medical  personnel),  the  public  (civilians),  and  the 
danger  to  the  public  (terrorists).  Police  are  separated  into  several  different  sides, 
representing  police  from  different  districts.  The  terrorists  are  also  divided  into 
two  sides:  those  that  incite  panic  and  those  that  attack  the  first  responders. 
Table  6  highlights  the  different  sides  and  their  relationship  to  each  other. 


Unit 

Friend 

Enemy 

1 .  Inner  Harbor  Police 

1 

1-6 

7-8 

2.  Traffic  Police 

2 

1-6 

7-8 

3.  SWAT 

3 

1-6 

7-8 

4.  Central  District  Police 

4 

1-6 

7-8 

5.  Fire  Departments 

5 

1-6 

7-8 

6.  Hospital 

6 

1-6 

7-8 

7.  Terrorist  Agitators 

7,8 

7,8 

1-6 

8.  Terrorist  Gunman 

7,8 

7,8 

1-6 

Table  6.  Relationship  Between  Sides 


f.  Sensors 

In  Pythagoras,  a  sensor’s  ability  to  detect  an  agent  is  based  on 
three  factors:  the  detection  ability  of  the  sensor,  the  terrain’s  concealment  factor, 
and  the  agent’s  detectability.  Each  of  these  factors  occurs  over  three  bands: 
A,  B,  and  C.  These  three  bands  enable  the  modeler  to  simulate  three  different 
types  of  detection,  e.g.,  visual  spectrum,  thermal  detection,  and  infrared.  A  piece 
of  terrain  may  deny  the  agent  an  ability  to  see  with  the  naked  eye,  but  visibility  is 
possible  with  an  infrared  or  thermal  device. 

In  this  simulation,  agents  have  one  sensor,  eyes,  which  detect  in 
the  visual  spectrum.  All  agents  are  completely  detectable.  The  limiting  factor  is 
the  concealment  factor  of  the  terrain,  previously  outlined  in  Table  2.  Figure  8 
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shows  the  effectiveness  of  eyes  by  range,  across  terrain  with  no  concealment 
factor.  To  find  the  actual  probability  of  detection,  use  the  following  formula: 


Actual  PD  =[1- (Concealment  Factor)]  *  Baseline  PD 


Figure  8.  Baseline  Probability  of  Detecting  a  Target  Using  Eyes 
g.  Communication  Devices 

Communication  devices  are  available  for  agents  to  pass 
information  to  each  other.  In  general,  there  are  two  major  types  of 
communication  devices:  those  that  require  line  of  sight  and  those  that  have 
broadcast  capability  and  therefore  do  not  require  line  of  sight.  Pythagoras 
enables  modelers  to  vary  the  probability  of  successful  communication  with  a 
given  device.  By  changing  the  probability  of  successful  communication,  the 
analyst  can  make  an  assessment  concerning  the  importance  of  a  solid 
communication  network  in  a  given  situation. 

In  this  simulation,  each  agent  is  given  a  voice  and  a  line  of  sight 
communication  device.  See  Figure  9  for  the  probability  of  successful 
communication  for  an  agent  using  its  voice.  Each  police,  fire,  and  medical  unit 
has  a  broadcasting  communication  device  that  simulates  the  handheld  radio 
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commonly  carried  by  each  agency  in  their  daily  duties.  The  handheld 
communication  device  is  capable  of  communicating  over  the  entire  simulation 
area. 


Figure  9.  Probability  of  Successful  Communications  Using  Voice 
h.  Agent  “Personalities” 

A  definition  for  personality  is  “the  total  of  qualities  and  traits,  as  of 
character  and  behavior,  that  are  peculiar  to  a  specific  person.”38 

Agent  personalities  in  this  MAS  result  from  the  effects  of  their 
behaviors  on  the  surrounding  environment,  including  other  agents.  Specifically, 
the  types  of  behaviors  that  are  used  to  define  agent  personalities  are:  the  desire 
to  hold  fire  in  the  presence  of  enemies  (aggression),  leadership  and  obedience, 
movement  desires,  and  alternate  behaviors  that  are  triggered  by  a  certain  action 
or  condition. 

(1)  Hold  Fire  Desire.  In  Pythagoras,  the  modeler  controls  an 
agent’s  hold  fire  desire  by  assigning  the  agent  a  number  on  a  scale  of  0  to  1 .  A 

38Definition  of  Personality.  Retrieved  on  10  May  2006  from  the  World  Wide  Web  at 
http://www.dictionarv.com 
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hold  fire  desire  of  1  is  analogous  to  restrictive  rules  of  engagement  (ROE),  in 
which  the  agent  will  not  fire  unless  fired  on.  A  hold  fire  desire  of  0.5  means  that 
half  the  time  when  an  agent  sees  an  enemy,  that  agent  will  fire  first,  and  half  the 
time  that  agent  will  not  fire  unless  fired  on.  A  hold  fire  desire  of  0  is  analogous  to 
a  “free-fire  zone”  in  which  the  agent  will  shoot  as  soon  as  an  enemy  is  identified. 
When  the  decision  to  engage  has  been  made,  the  agent  must  decide  from  one  of 
possibly  three  weapons.  The  modeler  must  identify  if  the  agent  will  select  the 
weapon  with  the  highest  probability  of  kill,  the  lowest,  or  the  medium  Pk. 

(2)  Leadership  and  Obedience.  Pythagoras  enables  the 
creator  of  the  simulation  to  incorporate  leadership  and  obedience  into  the  agents’ 
personality  profiles.  There  are  two  different  types  of  leaders  in  Pythagoras: 
charismatic  and  hierarchical.  When  exposed  to  charismatic  leadership,  the 
agents  within  the  unit  will  respond  to  the  leader  with  the  highest  leadership  value, 
a  value  between  0  and  100.  In  hierarchical  leadership,  the  agent  will  follow  the 
leader  that  has  the  smallest  leadership  value  higher  that  its  own  leadership 
value.  Agents  follow  their  leadership  in  accordance  with  their  obedience  value, 
established  by  the  simulation’s  creator  on  a  scale  of  0  to  1.  An  agent  with  an 
obedience  value  of  0.5  will  only  follow  orders  half  of  the  time. 

Leadership  and  obedience  each  have  a  “tolerance” 
associated  with  them.  This  tolerance  provides  a  way  for  the  modeler  to  capture 
the  effect  of  people  interpreting  situations  differently.  The  tolerance  is  on  the 
same  scale  as  the  initial  trait.  A  small  tolerance  would  indicate  a  predictable, 
disciplined  outcome.  A  large  tolerance  indicates  more  randomness,  with  possibly 
less  discipline  and  less  predictability. 

(3)  Movement  Desires  and  Alternate  Behaviors.  The 
developers  of  Pythagoras  provide  the  modeler  with  several  distinct  movement 
desires  that  help  to  shape  each  agent’s  simulated  personality.  Figure  9  provides 
a  depiction  of  available  movement  desires.  Each  movement  has  a 
corresponding  desire  and  desire  tolerance.  The  modeler  decides  which  desires 
to  activate  within  each  agent.  These  desires  are  adjusted  by  the  modeler  until 
the  agents  move  as  desired.  Every  desire  is  assigned  a  number  between 
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0  and  100.  An  agent  with  a  high  movement  desire  is  more  likely  to  execute  that 
behavior.  Event  movement  desire  has  a  tolerance  associated  with  it,  similar  to 
leadership  and  obedience. 

In  addition  to  the  desire  fields,  there  are  movement  desires 
that  correspond  to  a  desire  to  move  away  from  or  toward  other  agents.  These 
desires  are  based  on  counts  or  ratios.  For  example,  an  agent  may  have  the 
desire  to  move  away  from  an  enemy  if  the  force  ratio  is  greater  than  a  certain 
number.  Similar  desires  correspond  to  distances.  An  agent  may  have  the  desire 
to  move  toward  the  nearest  unit  member  if  that  unit  member  is  farther  away  than 
a  certain  threshold.  The  movement  desire  tab  enables  the  modeler  to  explicitly 
define  these  thresholds.  As  with  the  desired  tolerance,  the  modeler  can  implicitly 
capture  a  measure  of  discipline  or  organization  by  using  the  tolerances 
associated  with  ratios  or  distances. 

The  agent  “decides”  which  desire  to  select  by  virtue  of  a 
choice  the  modeler  makes,  illustrated  in  Figures  10  and  11.  The  agent  can  move 
in  accordance  with  its  highest  desire.  It  can  also  select  a  desire  randomly,  based 
on  the  draw  of  a  random  number.  In  addition,  the  decision  may  be  made  by 
calculating  a  weighted,  vector  average  of  the  active  desires.  Finally,  the  modeler 
may  decide  to  enable  the  agent  to  move  in  accordance  with  a  weighted  average 
of  its  top  two  desires. 
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Figure  10.  Pythagoras  Movement  Desires 

Behaviors  define  agents’  personalities  in  a  MAS.  As  the 
simulation  progresses,  the  agents  respond  to  stimuli  in  their  environment  in 
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different  ways.  These  alternate  behaviors  define  the  agents’  personalities  as 
much  as  the  behaviors  activated  at  the  beginning  of  the  simulation.  The  author 
of  the  simulation  creates  alternate  behaviors  to  capture  an  effect  that  occurs 
within  the  simulation.  For  example,  at  timestep  one,  when  the  bomb  explodes  in 
this  simulation,  it  triggers  the  civilians  to  panic.  The  specific  trigger  is  the 
timestep;  at  timestep  one,  the  properties  of  the  civilians  change  to  reflect  feeling 
the  effects  of  the  bomb,  then  panicking.  There  are  several  event  triggers 
provided  in  Pythagoras,  some  examples  of  which  are  illustrated  in  Table  7. 


Trigger 

Trigger 

Event 

Value 

Alternate  Behavior 
Triggered 

Alternate  Behavior 
Ordered 

Absolute  Timestep 

60 

EMTs  travel  to  site 

N/A 

Know  About  Enemy 

1 

N/A 

Kill  Enemy 

Arrive  at  Objective 

N/A 

EMTs  evac  wounded 

N/A 

Relative  Timestep 

450 

Fire  Dept,  leaves  rubble 

N/A 

N/A  =  Not  Applicable. 


Table  7.  Example  Alternate  Behavior  Triggers 

The  author  defined  the  panic  behavior  by  changing  certain 
qualities  of  the  agents  affected,  including  rate  of  movement,  movement  desires, 
and  a  establishing  a  need  to  be  “calmed.”  See  Table  8  for  a  list  of  alternate 
behaviors  developed  for  this  thesis.  Behaviors  of  each  individual  class  of  agent 
in  the  simulation  will  be  discussed  later  in  this  chapter,  in  each  respective 
section,  one  for  each  class. 
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Alternate  Behavior 
Name 

Agent  Class 
Affected 

Behavior  Description 

Feel  Effects  of  Bomb 

All  civilians 

Precursor  to  panic 

Panic 

All  civilians 

Increase  speed,  need  calm,  avoid  enemy 

Wait  for  Medic 

Stretcher  civ 

Stay  in  place  until  medic  comes 

Go  to  Medical  Station 

All  civilians 

Move  to  established  triage  point 

Move  to  Rubbling  Area 

Fire  Depts 

Deploy  from  home  fire  stations 

Evacuate  from  Rubble 

Fire  Depts 

Provide  aid  to  civilians  in  rubble 

Travel  from  Hospital 

HH 

Move  from  Mercy  Medical  and  St.  Johns 

Evac  Stretcher  Wounded 

Provide  aid,  bring  SW  civs  to  triage  point 

Attack  Responders 

Gunman 

Attack  civilians  and  first  responders 

Begin  Incident  Command 

Unit  Leaders 

Incident  command  post  operational 

Kill  Enemy 

SWAT 

Command  post  dispatches  SWAT 

Building  Clearing 

Police 

Follow-on  police  clear  affected  buildings 

Table  8.  List  of  Alternate  Behaviors 


Now  that  the  overall  setting  for  the  simulation  model  is 
established,  the  following  section  will  provide  information  about  the 
representation  of  the  individual  agents  defined  in  the  simulation.  The  section  is 
arranged  by  agent  type,  describing  the  following  agents  in  detail: 

•  The  bomb  detonated  in  the  amphitheater 

•  Civilians:  Stretcher  wounded  and  those  that  are  “walking  wounded” 
or  physically  unaffected  by  the  bomb 

•  Terrorists:  Covert  agents  that  incite  panic,  gunmen  that 

aggress  responders 

•  Police:  Traffic  police,  patrolmen,  follow-on  responders,  incident 
commanders,  and  the  SWAT  team 

•  Firemen  from  two  departments 

•  Medical  personnel  from  two  different  hospitals 

For  all  agents  except  the  bomb,  the  author  will  describe  the 
agent  class’s  activities  in  plain  language,  including  a  diagram  that 
highlights  transition  points  between  behavior  states  when  necessary  (see 
Appendix  A  for  more  detail).  A  description  of  the  Pythagoras  modeling  of 
the  agent  class’s  behavior  follows  the  plain  language  discussion. 
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2. 


The  Bomb 


The  bomb  that  explodes  during  the  Celebrate  Baltimore  festival  is  a  device 
approximately  the  same  size  as  the  bomb  that  destroyed  the  Murrah  Federal 
Office  building  in  Oklahoma  City  in  1995.  It  is  the  equivalent  of  approximately 
7,350  pounds  of  ammonium  nitrate,  in  twenty-two  55-gallon  drums.39  The  size  of 
the  bomb  is  assumed  before  the  simulation  begins  and  does  not  change.  This 
assumption  is  required  because  the  terrain  in  Pythagoras  is  static.  When  the 
simulation  begins,  the  terrain  is  fixed;  there  is  no  mechanism  by  which  the 
modeler  can  enable  the  strength  of  the  bomb  blast  to  automatically  alter  the 
terrain.  If  the  modeler  sought  to  analyze  different  bomb  strengths,  different  areas 
of  rubbling  must  be  built. 

a.  Cookie  Cutter  Damage  Function 

Damage  inflicted  on  people  within  the  rubbling  area  is  assessed 
using  a  cookie  cutter  damage  function.  There  is  no  partial  damage  assessment 
resultant  from  the  cookie  cutter  damage  function;  an  agent  within  the  effective 
radius  is  killed  or  not  killed,  based  on  the  probability  of  kill  associated  with  that 
weapon.  The  author  chose  to  use  the  cookie  cutter  damage  function  for 
modeling  the  close-in  damage  caused  by  the  bomb  to  capture  the  effect  of  a 
person  possibly  being  shielded  by  a  support  beam  or  other  covering  object,  and 
not  being  hurt  by  the  blast.  It  is  possible  that  an  agent  survives  the  cookie  cutter 
blast  effect,  but  is  damaged  by  the  Carleton  damage  effect  outlined  below.  Note: 
The  two  blast  radii  overlap.  See  Figure  1 1  for  the  blast  radii  associated  with  the 
cookie  cutter  and  Carleton  damage  functions. 


39E-mail  from  Dr.  Julie  Seton,  EPiCS  Program  Manager,  TRAC-WSMR,  titled  “Message  from 
Dan  Edmonson  -  CT  Incident  Details  for  TRAC-Monterey  Project,”  24  January  2006, 
office  communication. 
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Figure  11.  Illustration  of  Blast  Radii 
b.  Carleton  Damage  Function 

Damage  caused  to  people  outside  the  rubbling  range  is  assessed 
using  the  Carleton  damage  function,  a  well-documented  method  for  calculating 
damage  done  by  an  indirect  fire  weapon.  The  author  chose  the 
Carleton  damage  function  to  capture  the  effect  of  decreased  damage  to  the 
people  that  are  far  away  from  the  blast.  Figure  12  illustrates  the  relationship 
between  probability  of  kill  (Pk)  and  distance  with  the  Carleton  damage  function. 
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Figure  12.  Probability  of  Kill  Using  the  Carleton  Damage  Function 

A  large  explosion  will  cause  psychological  effects,  in  addition  to  the 
physical  effects  felt  by  people  within  the  blast  radius.  The  psychological  effects 
are  not  limited  to  people  directly  (physically)  affected  by  the  bomb;  panic  in  a 
large-scale  crisis  event  can  be  pervasive  throughout  the  area.  The  technique  for 
modeling  panic  in  the  civilian  population  follows  in  the  next  section. 

3.  Civilians 

In  a  crisis  situation,  the  civilians  involved  can  be  regarded  as  two  separate 
classes:  those  that  can  move  by  themselves  (physically  unharmed  or 

“walking  wounded”)  and  those  that  cannot  (“stretcher  wounded”).  In  this 
simulation,  the  two  are  modeled  differently.  Within  the  two  classes  of  civilians, 
men,  women,  boys,  and  girls  are  modeled  with  different  characteristics. 
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a.  Ambulatory  Wounded  and  Uninjured  Civilians 

At  the  beginning  of  the  simulation,  all  civilians  wander  randomly 
through  the  Celebrate  Baltimore  area  in  Baltimore’s  Inner  Harbor.  After  the 
explosion  of  the  bomb,  civilians  that  are  ambulatory  wounded  or  physically 
unharmed  follow  the  events  depicted  in  Figure  13. 


Figure  13.  Flow  Chart  of  Uninjured  and  Ambulatory  Wounded 

Civilian  Activity 

At  timestep  one,  the  bomb  explodes  in  the  Baltimore  Inner  Harbor 
Amphitheater.  All  of  the  civilians  in  the  area  panic,  including  those  that  are 
uninjured,  slightly  injured,  and  seriously  wounded.  If  a  person  is  slightly  injured, 
one  of  the  emergency  first  responders  must  stop,  render  first  aid,  and  provide 
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directions  to  the  medical  triage  point  that  is  being  set  up.  Through  the  interaction 
between  first  responder  and  civilian,  the  civilian  is  calmed  down  and  begins  to 
move  to  the  triage  point.  If  the  civilian  is  uninjured  in  the  bomb  blast,  their  feeling 
of  panic  is  less  than  someone  who  was  injured.  To  assuage  their  feeling  of 
panic,  a  first  responder  merely  needs  to  spend  30  seconds  calming  them  down 
and  providing  directions  to  a  triage  point  at  one  of  three  locations  outside  the 
area  affected  by  the  blast. 

Civilians  will  continue  to  panic  until  a  first  responder  provides  them 
with  the  level  of  comfort  that  they  need.  As  civilians  move  to  the  triage  point,  it  is 
possible  that  a  terrorist  agitator  encounters  them  and  causes  them  to  panic 
again.  If  this  happens,  a  first  responder  must  identify  the  newly  panicking 
civilians  and  calm  them  again.  If  a  civilian  has  not  been  calmed  by  a 
first  responder  and  a  crowd  agitator  finds  them  first,  it  will  take  the  first  responder 
longer  to  calm  the  civilian  and  put  that  person  on  the  path  to  the  triage  point. 

(1)  Common  Qualities.  All  civilian  agents  have  eyes  that  are 
instantiated  as  sensors  and  voice  as  a  communications  device,  as  previously 
stated.  Civilians  do  not  carry  weapons. 

(2)  Movement  Rates.  See  Table  9  for  a  translation  of 
Pythagoras-specific  terms  into  values  that  are  more  familiar  to  us.  The 
movement  rates  and  the  sensing  information  are  data  derived  from  AMSAA. 
When  calm,  civilians  move  at  an  average  speed  of  4  kilometers  per  hour  (kph),  or 
at  about  the  pace  of  an  average  walk.  The  maximum  speed  that  a  civilian  can 
move  while  calm  is  16  kph,  or  approximately  running  speed.  When  panicked, 
civilians  have  an  average  speed  of  1 1  kph,  with  a  maximum  speed  of  22  kph, 
which  is  a  sprint.  Appendix  A,  Section  A.I.e.  includes  a  snapshot  of  the 
spreadsheet  used  to  develop  movement  rates  throughout  this  research. 
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Characteristic 

Value 

(Pythagoras) 

Tolerance 

(Pythagoras) 

Value 

(Actual) 

Tolerance 

(Actual) 

Movement  Rate  (Calm)* 

7 

22 

4  kph 

12  kph 

— agam— 

20 

20 

11  kph 

11  kph 

Sense  and  Detect* 

Eyes 

Communication 

Voice 

Weapons 

None 

Vulnerability  (Man) 

0.5 

0 

Vulnerability  (Woman) 

0.75 

0 

Vulnerability  (Boy  and  Girl) 

1.0 

0 

‘Derived  from  AMSAA  data. 


Table  9.  Uninjured  and  Ambulatory  Wounded  Civilian  Characteristics 

(3)  Civilian  Vulnerability.  Vulnerability  corresponds  to  the 
amount  of  protection  an  agent  has  from  its  environment,  e.g.,  a  police  officer’s 
body  armor.  In  this  simulation,  men,  women,  and  children  have  different 
vulnerability  levels.  The  specific  assignment  of  vulnerability  levels  is  not 
scientific.  It  is  a  demonstration  that  it  is  possible  to  assign  characteristics  to 
civilians  that  can  highlight  differences  between  men,  women,  and  children  in  this 
situation.  For  this  simulation,  men  have  half  the  vulnerability  value  of  children. 
Women  are  halfway  between  men  and  children. 

(4)  Alternate  Behaviors.  Civilians  of  this  class  have  two 
alternate  behaviors:  feeling  the  psychological  effects  of  the  bomb  and  panic. 

(a)  Feeling  Psychological  Effects  of  the  Bomb.  When 
the  bomb  explodes,  civilians  hear  the  bomb  and  they  become  afraid.  With  the 
fear  they  feel,  the  civilians  also  gain  a  need  to  be  calmed.  In  simulation,  the 
author  modeled  fear  with  Pythagoras’s  “attribute  alpha.”  When  the  bomb 
explodes,  every  civilian  is  given  10  units  of  attribute  alpha.  Civilians  that  are 
physically  affected  by  the  bomb  receive  20  additional  units  of  attribute  alpha. 
Ten  units  of  attribute  alpha  are  greater  than  the  threshold  that  causes  panic,  so 
the  panic  behavior  is  triggered. 

In  addition  to  fear,  the  explosion  awakens  a  need  for 
the  civilians  to  be  calmed.  In  this  simulation,  calm  is  modeled  by  Pythagoras’s 
“resource  X.”  Civilians  are  considered  consumers  of  this  resource;  police,  fire, 
and  medical  personnel  are  modeled  as  suppliers  of  the  resource. 
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(b)  Panic.  When  civilians  panic,  their  speed  increases 
and  they  become  aware  of  enemies  (terrorists).  Civilians  want  to  run  away  from 
any  terrorists  they  see  and  will  continue  to  panic  until  their  fear  is  removed  and 
their  calm  is  restored  by  police,  fire,  or  medical  personnel.  First  responders  can 
remove  fear  (attribute  alpha)  by  giving  orders  to  move  toward  the  triage  points. 
These  “orders”  are  a  paintball  weapon  that  decrease  attribute  alpha.  In  addition 
to  taking  away  fear,  first  responders  are  suppliers  of  calm  (resource  X).  When 
civilians  are  supplied  with  more  than  25%  of  calm  (resource  X)  and  have  less 
than  10  units  of  fear  (attribute  alpha),  another  alternate  behavior  is  triggered:  go 
to  medical  station. 

(c)  Go  to  Medical  Station.  When  civilians  cease 
panicking,  they  move  to  the  nearest  of  the  three  medical  stations  set  up  outside 
the  Celebrate  Baltimore  area.  Civilians  will  continue  to  move  to  a  medical  station 
unless  a  terrorist  interdicts  them  and  makes  them  panic  by  giving  the  civilians 
additional  fear  (attribute  alpha). 

It  is  important  to  understand  that  the  numbers 
associated  with  attribute  alpha  and  resource  X  have  been  chosen  to  produce 
certain  effects,  and  have  no  physical  significance.  The  numbers  are  not  as 
important  as  the  relationship  between  the  agents.  In  this  simulation,  police  and 
terrorists  have  the  same  level  of  effect  on  civilians.  Police  are  required  to  interact 
more  with  civilians  that  have  been  physically  affected  by  the  bomb.  Also,  police 
are  required  to  do  two  things  to  stop  civilians  from  panicking:  they  must  assuage 
the  civilians’  fear  and  calm  them  down. 

b.  Stretcher  Wounded  Civilians 

After  the  explosion  of  the  bomb,  civilians  that  are  stretcher 
wounded  follow  the  events  depicted  in  Figure  14. 
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Bomb  Explodes 


No 


Wait  for 
Medic 


Figure  14.  Flow  Chart  of  Stretcher  Wounded  Civilian  Activity 

At  timestep  one,  the  bomb  explodes  in  Baltimore’s  Inner  Harbor 
Amphitheater.  Stretcher  wounded  civilians  remain  in  one  place  after  the  bomb 
explodes;  they  cannot  move  until  they  are  stabilized  by  medical  personnel. 
EMTs  come  to  the  scene  of  the  explosion  from  two  hospitals.  The  EMTs 
stabilize  the  stretcher  wounded  personnel  and  move  them  to  the  triage  point. 
Stretcher  wounded  civilians  are  not  affected  by  the  panic  exhibited  by  other 
civilians  because  they  are  being  carried  on  a  litter  and  cannot  otherwise  move. 

(1)  Common  Qualities.  All  civilian  agents  have  eyes 
instantiated  as  sensors  and  voice  as  a  communications  device,  as  previously 
stated.  Civilians  do  not  carry  weapons. 

(2)  Movement  Rates.  See  Table  10  for  a  translation  of 
Pythagoras-specific  terms  into  values  that  are  more  familiar  to  us.  The 
movement  rates  and  the  sensing  information  are  data  derived  from  AMSAA.  The 
average  speed  that  a  stretcher  wounded  civilian  can  move  after  being  stabilized 
is  about  1  kph,  or  about  the  speed  that  a  Soldier  walks  while  under  a  heavy 
combat  load. 
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Characteristic 

Value 

Tolerance 

Value 

Tolerance 

(Pythagoras) 

(Pythagoras) 

(Actual) 

(Actual) 

Movement  Rate* 

2 

3 

1  kph 

1.5  kph 

Sense  and  Detect* 

Eyes 

Communication 

Voice 

Weapons 

None 

Vulnerability  (Man) 

0.5 

0 

Vulnerability  (Woman) 

0.75 

0 

Vulnerability  (Boy  and  Girl) 

1.0 

0 

‘Derived  from  AMSAA  data. 

Table  10.  Stretcher  Wounded  Civilian  Characteristics 

(3)  Civilian  Vulnerability.  In  this  simulation,  men,  women, 


and  children  have  different  vulnerability  levels.  The  specific  assignment  of 
vulnerability  levels  is  not  scientific.  It  is  a  demonstration  that  it  is  possible  to 
assign  characteristics  to  civilians  that  can  highlight  differences  between  men, 
women,  and  children  in  this  situation.  For  this  simulation,  men  have  half  the 
vulnerability  value  of  children.  Women  are  halfway  between  men  and  children. 

(4)  Movement  Desires.  Civilians  of  this  class  have  one 
desire:  move  toward  the  medical  triage  point.  Although  the  civilians  want  to  get 
to  the  triage  point,  they  cannot  begin  moving  in  that  direction  until  visited  by  a 
medic.  A  stretcher  wounded  civilian’s  health  is  simulated  by  fuel.  When  the 
bomb  explodes,  they  run  out  of  fuel,  so  they  cannot  move.  The  medics  from  the 
hospital  are  fuel  providers.  It  is  only  through  an  interface  between  the  medics 
and  the  stretcher  wounded  civilians  that  the  civilians  are  stabilized  (receive  their 
fuel)  and  are  able  to  move  to  the  triage  point. 

4.  Terrorists 

a.  Agitators 

(1 )  Priority  of  Work.  Terrorist  agitators  have  one  goal:  incite 
panic  in  the  crowd  after  the  explosion  of  the  bomb.  The  agitators  hide  in 
buildings  until  after  the  explosion,  then  come  out  of  the  buildings  and  interact 
with  the  crowd.  The  agitators  do  not  focus  on  individual  people,  but  attempt  to 
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cover  as  much  ground  as  possible  in  the  response  zone,  thereby  maximizing 
their  influence. 


(2)  Common  Qualities.  All  terrorist  agents  have  eyes 
instantiated  as  sensors  and  voice  as  a  communications  device. 

(3)  Movement  Rates.  Movement  rates  and  the  sensing 
information  are  data  derived  from  AMSAA.  Terrorists  move  at  an  average  speed 
of  4  kph,  or  at  about  the  pace  of  an  average  walk.  The  maximum  speed  that  a 
terrorist  can  move  is  16  kph,  or  approximately  running  speed. 

(4)  Weapon.  Terrorist  agitators  have  a  paintball  weapon  that 
increases  the  target’s  level  of  attribute  alpha,  which  corresponds  to  fear  in  this 
simulation.  One  shot  of  the  weapon  provides  enough  fear  to  cause  the  civilians 
to  panic. 

(5)  Simulating  Covert  Operation.  By  using  the  ability  of 
agents  in  Pythagoras  to  change  colors,  it  was  possible  to  enable  the  agitators  to 
“hide”  in  the  crowd.  Table  1 1  illustrates  the  use  of  color  as  a  “hiding”  mechanism 
for  the  agitators.  At  the  beginning  of  the  simulation,  the  agitators  consider 
everyone  to  be  an  enemy;  however,  the  civilians  and  police  consider  the 
agitators  to  be  friendly.  As  an  agitator  starts  to  cause  panic,  he  gradually 
changes  color.  After  five  shots,  first  responders  and  civilians  are  able  to  identify 
the  agitator  as  an  enemy.  This  gradual  color  change  provides  the  effect  that  it 
may  take  the  police  some  time  to  determine  who  is  causing  trouble. 


Red 

Green 

Blue 

Civilians 

Hospital 

Police 

Agitator  (Start) 

155 

150 

0 

Friend 

Friend 

Friend 

Agitator  (after  1st  shot) 

hb 

120 

0 

mmm 

Friend 

Agitator  (after  2nd  shot) 

mm 

90 

0 

mmm 

Friend 

Agitator  (after  3rd  shot) 

185 

60 

0 

Friend 

Friend 

Neutral 

Agitator  (after  4th  shot) 

195 

30 

0 

Neutral 

Friend 

Neutral 

Agitator  (after  5th  shot) 

iflUi 

0 

0 

Enemy 

Enemy 

Enemy 

Civilians 

0 

205 

0 

Friend 

EB 

Hospital 

0 

220 

mm 

Friend 

EB 

Police 

0 

220 

mm 

Friend 

Friend 

Table  11.  Terrorist  Agitator  Color  Change 
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Read  Table  1 1  as  follows:  The  agent  in  the  column  sees  the  agent 
in  the  row  in  the  manner  of  the  intersecting  cell.  For  example,  Civilians  see 
Agitator  as  a  friend  at  the  start  of  the  simulation.  Civilians  see  Agitator  as  neutral 
after  shot  four.  Civilians  see  Agitator  as  an  enemy  after  shot  five. 

b.  Gunman 

(1)  Personality  and  Priority  of  Work.  The  terrorist  gunman’s 
goal  supports  the  agitator  goal  of  causing  panic.  The  gunman  will  hide  in  a 
building  until  the  first  responders  have  arrived  on  the  scene,  then  attack  first 
responders  and  civilian  targets  of  opportunity. 

(2)  Common  Qualities.  All  terrorist  agents  have  eyes 
instantiated  as  sensors  and  voice  as  a  communications  device. 

(3)  Movement  Rates.  Movement  rates  and  the  sensing 
information  are  data  derived  from  AMSAA.  When  calm,  terrorists  move  at  an 
average  speed  of  4  kph,  or  at  about  the  pace  of  an  average  walk.  The  maximum 
speed  that  a  terrorist  can  move  is  16  kph,  or  approximately  running  speed. 

(4)  Weapon.  The  gunman  carries  an  AK-47  Assault  Rifle. 
Weapon  capabilities  were  derived  from  data  supplied  by  AMSAA. 

(5)  Terrorist  in  Hiding.  The  gunman  is  able  to  hide  by 
programming  his  base  behavior  so  he  is  completely  concealed,  does  not  move, 
and  therefore  is  undetectable.  After  the  first  responders  are  involved  in  tending 
to  the  situation  (approximately  timestep  375;  25  minutes  in  actual  time),  a 
behavior  is  triggered  in  the  gunman  to  attack  emergency  services.  During  his 
attack,  he  is  visible,  armed,  and  has  the  desire  to  attack  the  enemy,  but  not  get 
too  close.  This  rule  of  engagement  simulates  the  effect  of  a  sniper. 

5.  Police 

This  section  includes  modeling  decisions  that  reflect  the  author’s 
determination  of  possible  priorities  of  work  for  emergency  first  response 
organizations.  Although  the  author  has  experience  in  crisis  management 
planning  at  the  military  installation  level,  he  established  priorities  of  work  entirely 
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without  the  use  of  Baltimore  standing  operating  procedures,  or  consultation  with 
Baltimore  officials.  This  simulation  is  not  intended  to  specifically  replicate 
Baltimore  emergency  response.  TTPs  may  exist  that  are  more  effective  than 
those  established  in  this  simulation.  The  purpose  of  this  thesis  is  not  to 
demonstrate  the  author’s  knowledge  of  crisis  response  TTPs,  but  rather  to  show 
that  it  is  possible  to  model  reasonable  emergency  response  procedures  in  a 
multi-agent  environment. 

In  this  simulation,  police  forces  share  similar  characteristics.  All  police 
units  except  the  SWAT  team  have  the  same  movement  rates,  sensory  ability, 
weapons,  and  body  armor.  The  author  will  detail  this  information  for  the  traffic 
police  in  Section  5. a.,  but  will  not  repeat  it  for  subsequent  units.  Communication 
ability  is  similar  between  units.  Each  unit  uses  a  communication  device  that 
simulates  a  handheld  radio.  The  members  of  each  unit  can  communicate  with 
each  other,  but  cannot  communicate  with  members  of  other  units  via  handheld 
radio.  Members  of  different  units  can  talk  to  each  if  they  are  close  enough  to  use 
the  voice  communication  device. 

a.  Traffic  Police  at  Intersections 

(1)  General  Characteristics.  Table  12  describes  the 


characteristics  of  the  police  that  are  stationed  at  the  intersections  of  major  roads 
leading  to  the  Celebrate  Baltimore  area. 


Characteristic 

Value 

(Pythagoras) 

Tolerance 

(Pythagoras) 

Value 

(Actual) 

Tolerance 

(Actual) 

Movement  Rate* 

7 

7 

4  kph 

4  kph 

Sense  and  Detect* 

Eyes 

Communication 

Handheld  Radio  and  Voice 

Weapons 

9mm  Pistol,  Medical  Kit,  Orders  to  Civilians 

Body  Armor 

Level  lll-A:  Most  Handguns,  Some  Small  Rifles 

*Derived  from  AMSAA  data. 

Table  12.  Characteristics  of  Traffic  Police  at  Intersections 

(2)  Personality  and  Priorities  of  Work.  Figure  15  illustrates 
the  priorities  of  work  for  traffic  police. 
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Traffic  Policeman  Assumes 
Duty  at  Intersection 


Figure  1 5.  Priorities  of  Work  for  T raffic  Police  at  Intersections 

The  author  assumes  that  traffic  police  are  on  duty  at  the 
intersections  leading  into  the  Celebrate  Baltimore  area  throughout  the  festival; 
that  is,  they  are  in  position  when  the  bomb  explodes.  In  the  simulation,  traffic 
police  maintain  position  in  the  immediate  vicinity  of  their  intersection,  ensuring 
the  crisis  event  is  contained  and  chaos  does  not  spread  outside  the  immediate 
area.  If  traffic  police  see  an  enemy,  they  will  alert  the  other  members  of  the  unit, 
attack  the  enemy,  and  neutralize  the  threat.  When  the  threat  is  neutralized,  the 
traffic  police  will  then  return  to  their  initial  intersection. 

(3)  Weapons  and  Body  Armor.  Police  carry  three  tools  that 
they  can  use  to  influence  the  agents  around  them. 

(a)  9mm  Semiautomatic  Pistol.  The  9mm  pistol,  in 
various  forms,  is  the  most  commonly  carried  pistol  in  police  units.  The  author 
used  Army  Field  Manual  3-23.35  ( Combat  Training  with  Pistols,  M9  and  Mil), 
the  experience  of  subject  matter  experts,  and  personal  experience  as  a 
military  policeman  to  generate  the  characteristics  of  this  weapon.  Police  are  able 
to  fire  one  shot  using  the  9mm  every  three  timesteps,  amounting  to  one  aimed 
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shot  every  12  seconds  of  actual  time.  Maximum  engagement  range  is 
approximately  82  pixels  (50  meters).40  Forty-five  rounds  is  a  common  standard 
load  for  police  to  carry  in  a  large  city.  A  random  damage  degree  of  0.25  means 
that  75%  of  the  damage  assessed  to  a  target  will  be  assessed  deterministically, 
using  the  weapon’s  probability  of  kill.  Twenty-five  percent  of  the  damage 
assessed  is  stochastic,  based  on  a  random  number  draw.  Figure  16  shows  the 
weapon’s  single  shot  probability  of  kill  (SSPK),  derived  by  multiplying  the 
probability  of  hit  and  probability  of  kill,  given  a  hit. 


Single  Shot  Probability  of  Kill  (SSPK)  Using  9mm 


^^“Single  Shot  Probability  of  Kill 


Figure  16.  9mm  Single  Shot  Probability  of  Kill 

(b)  Medical  Kit.  The  medical  kit  is  a  “restorative” 
weapon;  i.e.,  its  usage  increases  the  target’s  health.  Police  can  target  unit 
members,  friends,  and  neutral  agents  with  the  medical  kit.  When  police  use  the 
medical  kit  to  perform  first  aid  on  a  civilian,  the  policeman  is  able  to  reduce  the 
level  of  fear  the  civilian  agent  feels  (by  decreasing  the  level  of  attribute  alpha). 
For  the  medical  kit  to  be  effective,  the  administering  agent  must  be  within  a  close 
distance  to  the  target. 


40United  States  Army,  Field  Manual  3-23.35,  Combat  Training  with  Pistols,  M9  and  Mil, 
June  2003,  pp.  1-2. 
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(c)  Orders  from  Police.  Orders  are  a  “paintball” 
weapon  whose  usage  causes  a  decrease  in  the  target’s  “fear,”  i.e.,  a  decreased 
level  of  attribute  alpha.  Police  can  target  friends  and  neutral  agents  with  the 
orders  paintball  weapon.  Orders  are  effective  to  a  distance  of  100  feet 
(50  pixels). 

(d)  Body  Armor.  In  this  simulation,  all  police  wear 
body  armor  that  represents  armor  capable  of  stopping  most  handguns  and  some 
small  caliber  rifles.  This  body  armor  also  provides  protection  against  other 
threats,  such  as  blunt  force  and  slashing  trauma.  Body  armor  is  simulated  by 
reducing  police  vulnerability  by  50%. 

b.  Police  on  Patrol 

(1)  General  Characteristics.  Patrolmen  share  the  same 
general  characteristics  as  the  police  at  intersections  described  in  the  previous 
section.  Patrolmen  have  one  additional  capability:  they  are  a  supplier  of 
Pythagoras’s  “resource  X,”  a  measure  of  influence  that  will  be  described  in  detail 
later  in  this  same  section.  Each  patrol  consists  of  two  policemen. 

(2)  Personalities  and  Priorities  of  Work.  Figure  17  is  a 
diagram  of  the  patrol  areas  in  which  the  five  patrols  operate  during  the  scenario. 
Patrols  one  and  four  circulate  around  their  respective  areas  of  operation  in 
accordance  with  the  patterns  in  Figure  17.  Patrol  number  two  stays  in  the  area 
of  the  amphitheater  because  of  crowd  density.  Patrols  three  and  five  have  the 
same  route,  but  cover  it  at  different  times. 
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Figure  17.  Police  Patrol  Areas 

Figure  18  outlines  the  patrolmen’s  responsibilities  and 
priorities  of  work  throughout  the  simulation. 
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Figure  1 8.  Priorities  of  Work  for  Patrolmen 

In  the  case  of  police  patrolmen,  there  are  several  competing 
priorities;  the  agent  must  “decide”  which  priority  to  address.  In  Pythagoras,  the 
modeler  facilitates  this  decision  by  assigning  appropriate  weights  to  the 
movement  desires  that  correspond  with  a  given  priority.  In  this  scenario,  the 
patrolman’s  top  priority  is  to  calm  panicking  civilians,  thereby  restoring  calm 
throughout  the  affected  area.  Tending  to  civilians  that  are  already  injured  is  of 
secondary  importance.  By  calming  the  panicking  crowd,  the  patrolman  helps  to 
ensure  that  more  civilians  are  not  injured.  Enemy  engagement  is  of  tertiary 
importance  because  the  SWAT  team  is  alerted  to  engage  the  threat. 

In  this  simulation,  panicking  civilians  have  two 
characteristics  that  alert  the  police  of  their  state  of  panic: 
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•  Low  level  of  calm  (Pythagoras’s  resource  X) 

•  Raised  level  of  fear  (Pythagoras’s  attribute  alpha) 

Police  patrols  alleviate  both  of  the  above  civilian  conditions,  enabling  them  to  be 
relieved  of  their  state  of  panic  and  move  to  the  medical  triage  point. 

Patrolmen  are  set  up  in  Pythagoras  as  suppliers  of 
resource  X.  As  stated  earlier,  resource  X  in  the  simulation  represents  an 
objective  measure  of  an  agent’s  level  of  “calm.”  The  Pythagoras  model  enables 
consumers  of  a  given  resource  to  notify  suppliers  when  the  consumer  reaches  a 
certain  threshold,  established  in  advance  by  the  modeler.  Suppliers  receive  the 
resource  request  from  consumers  and  move  to  the  consumers  to  impart  the 
resource  needed.  In  this  simulation,  the  calmed  civilian  is  given  enough  “calm” 
for  the  entire  scenario,  if  not  attacked  by  a  terrorist.  If  the  civilian  is  attacked  by  a 
terrorist,  that  civilian  will  panic  again.  Reversion  to  the  panic  behavior  causes  a 
loss  of  the  resource,  reinitiating  the  call  for  more.  Patrolmen  regenerate  their 
supply  of  resource  X;  thus,  they  will  not  run  out  of  the  ability  to  calm  civilians. 

In  addition  to  the  ability  to  calm  civilians  using  resource  X, 
the  patrolmen  provide  directions  to  a  medical  triage  site  and  orders  to  go  there. 
These  orders  are  modeled  as  a  “paintball”  weapon,  an  influence  tool  that 
reduces  the  civilian’s  level  of  attribute  alpha  (fear).  A  civilian  is  considered  calm 
as  long  as  the  civilian: 

•  Maintains  a  resource  X  level  greater  than  25%  of  maximum  level 

•  Has  an  attribute  alpha  level  lower  than  the  amount  imparted  by  one 
terrorist  interaction 

If  there  are  no  panicking  civilians,  the  patrolman  will  attempt 
to  identify  wounded  civilians.  Upon  identification  of  a  wounded  person,  the 
patrolman  will  move  to  the  wounded  person  and  perform  first  aid.  In  this 
simulation,  first  aid  is  modeled  with  an  influence  tool  that  has  restorative 
properties.  When  engaged  with  this  influence  tool,  the  target’s  health  increases. 
After  the  target’s  health  is  restored,  the  patrolman  will  again  try  to  identify  if  a 
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civilian  needs  to  be  calmed.  If  not,  the  patrolman  will  seek  to  identify  another 
wounded  civilian. 

Patrolmen  will  engage  an  enemy  that  they  see  if  and  only  if 
their  primary  and  secondary  missions  have  been  fulfilled.  If  the  situation  is  such 
that  the  patrolman  must  engage  an  enemy  threat,  the  weapon  of  choice  is  the 
patrolman’s  9mm  pistol.  After  the  engagement,  the  patrolman  will  again  check  to 
see  if  a  primary  or  secondary  mission  must  be  fulfilled.  If  not,  and  there  are  no 
additional  enemies  in  sight,  the  patrolman  will  continue  along  the  predetermined 
patrol  route. 


c.  Follow-On  Police 

(1)  General  Characteristics.  Follow-on  police  are  police 
personnel  that  assemble  at  the  Central  District  headquarters  in  response  to  the 
first  alarm  initiated  by  Baltimore’s  central  dispatching  authority.  The 
characteristics  of  these  personnel  are  the  same  as  the  traffic  police  and  the 
patrolmen  that  were  on  station  when  the  bomb  exploded.  Follow-on  police  enter 
the  simulation  as  two  teams  of  three  policemen  each.  These  teams  arrive  at 
intervals  of  15  minutes  (225  timesteps)  after  the  explosion,  starting  at 
timestep  225. 

(2)  Priorities  of  Work.  First-alarm  police  have  similar  goals 
to  the  patrolmen  described  in  the  previous  section;  the  decision  template 
diagram  of  Figure  18  applies.  The  follow-on  patrols  seek  to  spread  a  calming 
influence  by  clearing  the  buildings  in  the  area  surrounding  the  bomb  blast  and 
directing  the  people  to  the  medical  triage  points.  Figure  19  identifies  one  route 
that  first-alarm  police  follow  to  provide  assistance  to  the  area  affected  by  the 
bomb.  The  route  shown  illustrates  a  west  to  east  path  that  is  taken  by  one  team 
of  three  personnel.  The  other  team  of  three  that  is  dispatched  at  the  same  time 
follows  the  same  path,  but  in  an  east  to  west  direction. 
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Figure  19.  West  to  East  Route  of  Follow-on  Police 
d.  Incident  Command  Post  (ICP) 

(1)  General  Characteristics.  The  ICP  is  located  in  the 
convention  center,  directly  to  the  west  of  the  amphitheater.  The  purpose  of  the 
ICP  is  to  provide  situational  awareness  to  the  entire  response  force.  It  is  through 
the  ICP  that  the  different  organizations  involved  in  the  response  know  what  each 
other  are  doing.  The  ICP  is  comprised  of  one  leader  from  each  of  the 
organizations  involved  in  the  simulated  response  to  this  crisis: 


•  Traffic  Police 

•  Initial  Response  Patrolmen 

•  Fire  Department 

•  Hospital 

•  Central  District  Police  (first-alarm,  follow-on  responders) 

•  SWAT  Team 


(2)  Incident  Command  Structure.  See  Figures  20-22  for 
diagrams  of  the  ICP’s  functionality. 
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Figure  20.  ICP  Receives  Information 


Figure  21.  Information  Sharing 


Figure  22.  ICP  Disseminates  Information 


The  leaders  that  comprise  the  ICP  receive  information  from 
the  members  of  their  individual  organizations,  using  a  communication  device  that 
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simulates  a  standard  handheld  radio.  The  ICP  is  a  well-connected  organization, 
meaning  that  each  member  of  the  ICP  has  communication  with  every  other 
member.  Members  of  ICP  communicate  internally  via  a  communication  device 
that  simulates  a  voice.  After  information  is  passed  throughout  the  ICP,  the 
leaders  send  the  consolidated  information  back  to  the  members  of  the 
organizations  that  are  conducting  the  crisis  response  operations.  In  this  manner, 
every  member  of  every  organization  has  situational  awareness  of  the 
operational  environment. 

One  example  of  the  importance  of  the  ICP  is  the 
identification  of  the  terrorist  gunman  in  this  scenario.  If  a  patrolman  encounters 
the  gunman,  he  is  at  a  disadvantage;  the  patrol  is  armed  with  a  pistol,  the 
gunman  with  an  AK-47.  The  patrolman  communicates  the  location  of  the 
terrorist  to  the  ICP.  The  ICP  transmits  that  location  to  the  SWAT  team,  which 
responds  with  nine  personnel  carrying  M4  carbines,  a  decided  advantage  for  the 
good  guys. 


e.  SWAT  Team 

The  SWAT  team  is  located  in  the  Central  District  headquarters, 
north  of  the  bomb  attack  location.  The  mission  of  the  SWAT  team  in  this 
simulation  is  to  receive  information  about  the  location  of  terrorists,  move  to  that 
location,  and  neutralize  the  terrorist  threat.  After  the  threat  has  been  neutralized, 
the  SWAT  team  returns  to  the  Central  District  headquarters  to  await  another  call. 
The  team  is  comprised  of  nine  police  officers.  These  officers  have  sensory  and 
communication  characteristics  that  are  similar  to  the  other  police  officers 
described  in  this  simulation.  Table  13  outlines  the  characteristics  of  SWAT  team 
members.  SWAT  team  members  have  the  capability  to  move  faster  than  other 
police  officers  and  they  are  more  heavily  armed.  SWAT  members  carry  a  rifle  in 
addition  to  their  service  pistol. 
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Characteristic 

Value 

(Pythagoras) 

Tolerance 

(Pythagoras) 

Value 

(Actual) 

Tolerance 

(Actual) 

Movement  Rate* 

7 

22 

4  kph 

12  kph 

Sense  and  Detect* 

Eyes 

Communication 

Handheld  Radio  and  Voice 

Weapons 

M4  Carbine,  9mm,  Medical  Kit 

Body  Armor 

Level  lll-A:  Most  Handguns,  Some  Small  Rifles 

*Derived  from  AMSAA  data. 


Table  13.  Characteristics  of  SWAT  Team 
f.  Firefighters 

(1)  General  Characteristics.  Firemen  are  initially  located  at 
two  fire  stations.  Fire  Station  23  is  located  approximate  eight-tenths  of  a  mile 
east  of  the  location  of  the  bomb  blast.  Fire  Station  33  is  located  about 
three-quarters  of  a  mile  south  of  the  blast.  Table  14  includes  the  general 
characteristics  of  the  firefighters  in  this  simulation.  In  addition  to  the  ability  to 
provide  first  aid  with  the  medical  kit  and  direct  civilians  to  the  triage  point, 
firefighters  have  the  ability  to  stabilize  stretcher  wounded  personnel  and  move 
them  to  the  triage  point.  The  mechanism  for  this  action  is  described  in  detail  in 


Section  IV.A.5.g,  which  discusses  medical  personnel. 


Characteristic 

Value 

(Pythagoras) 

Tolerance 

(Pythagoras) 

Value 

(Actual) 

Tolerance 

(Actual) 

Movement  Rate* 

7 

7 

4  kph 

4  kph 

Sense  and  Detect* 

Eyes 

Communication 

Handheld  Radio  and  Voice 

Weapons 

Medical  Kit,  Orders  to  Civilians 

Body  Armor 

None 

*Derived  from  AMSAA  data. 


Table  14.  Characteristics  of  Firefighters 

(2)  Priorities  of  Work.  Firefighters  deploy  to  the  bomb  attack 
area  five  minutes  (75  timesteps)  after  the  explosion  occurs.  Teams  of 
five  personnel  deploy  from  their  respective  fire  stations  directly  to  the  two 
buildings  that  sustain  rubbling  damage.  Fire  Station  23  moves  to  the  northern 
rubble  area.  Fire  Station  33  moves  to  the  southern  rubble  area.  The  firefighters’ 
mission  is  to  search  the  rubble  for  survivors  of  the  blast.  Upon  identification  of 
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survivors,  the  firefighters  have  the  capability  to  provide  the  level  of  medical 
support  that  a  survivor  needs,  from  first  aid  (using  the  medical  kit,  as  described 
earlier)  to  stabilization  of  stretcher  wounded  civilians  (described  in  detail  with 
discussion  of  hospital  personnel  in  the  next  section).  Upon  stabilizing  the 
civilians,  firemen  will  order  them  to  a  triage  point.  Through  the  interaction 
between  firefighters  and  civilians,  calm  is  restored  and  panic  reduced  in  the 
same  manner  as  with  police  interactions  described  earlier. 

g.  Medical  Personnel 

(1)  General  Characteristics.  Medical  personnel  are  initially 
located  at  two  hospitals.  Mercy  Medical  Center  is  located  approximately 
nine-tenths  of  a  mile  north  of  the  location  of  the  bomb  blast.  Saint  Johns  Hopkins 
Hospital  is  located  about  two  miles  northeast  of  the  blast.  Table  15  includes  the 
general  characteristics  of  the  medics  in  this  simulation.  In  addition  to  the  ability 
to  provide  first  aid  with  the  medical  kit  and  direct  civilians  to  the  triage  point, 
medics  have  the  ability  to  stabilize  stretcher  wounded  personnel  and  move  them 
to  the  triage  point. 


Characteristic 

Value 

Tolerance 

Value 

Tolerance 

(Pythagoras) 

(Pythagoras) 

(Actual) 

(Actual) 

Movement  Rate  to  Area* 

50 

16 

27  kph 

11  kph 

Movement  Rate  (Treatment) 

1 

0 

0.5  kph 

0 

Sense  and  Detect* 

Eyes 

Communication 

Handheld  Radio  and  Voice 

Weapons 

Medical  Kit,  Orders  to  Civilians 

Body  Armor 

None 

‘Derived  from  AMSAA  data. 


Table  15.  Characteristics  of  Medical  Personnel 

(2)  Treatment  of  Stretcher  Wounded  Civilians.  After  the 
bomb  explodes,  many  civilians  in  the  immediate  area  of  the  blast  are  critically 
wounded.  Their  wounds  are  serious  enough  that  they  cannot  be  stabilized  by  a 
policeman  with  standard  shift  equipment.  In  this  simulation,  only  a  trained  agent 
(from  a  hospital  or  fire  department)  has  the  training  and  equipment  needed  to 
stabilize  a  stretcher  wounded  civilian.  When  an  EMT  identifies  a  stretcher 
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wounded  civilian,  the  EMT  moves  to  the  civilian  and  provides  treatment. 
Following  the  treatment,  the  civilian  is  moved  to  the  triage  point. 

The  author  uses  Pythagoras’s  concept  of  fuel  to  simulate  the 
treatment  of  people  who  are  unable  to  move  due  to  their  injuries.  The  use  of  fuel 
is  similar  to  the  police  force’s  use  of  resource  X  to  restore  calm  in  panicking 
civilians.  Stretcher  wounded  civilians  cannot  move  because  they  are  consumers 
of  fuel  and  have  run  out.  In  this  simulation,  only  EMTs  are  suppliers  of  fuel,  so 
only  EMTs  can  effectively  treat  stretcher  wounded  civilians.  When  a 
stretcher  wounded  civilian  is  short  on  fuel,  that  agent  will  request  fuel  from  the 
supplier.  The  supplier  will  move  to  the  consumer  and  provide  the  fuel.  At  that 
time,  the  consumer  will  begin  moving  in  accordance  with  their  desires. 

There  is  no  mechanism  in  Pythagoras  to  simulate  one  agent 
carrying  another.  After  being  stabilized,  the  stretcher  wounded  civilians  move  to 
the  medical  triage  point  under  their  own  power;  the  speed  is  comparable  to  a 
soldier  who  is  moving  under  heavy  combat  load.  Speed  under  heavy  combat 
load  (2  kph)  is  approximately  the  speed  that  trained  personnel  would  carry  a 
stretcher.  To  simulate  the  EMT  carrying  the  stretcher,  the  speed  of  the  EMT  is 
reduced  to  only  0.5  kph.  If  the  EMT  was  actually  carrying  the  stretcher,  that 
agent  would  not  be  able  to  provide  treatment  to  another  patient  until  the  first 
patient  was  dropped  off  and  the  EMT  returned  to  the  scene.  This  effect  is 
accounted  for  in  the  simulation  because  the  EMT’s  movement  is  slowed 
dramatically;  i.e.,  due  to  the  slow  movement,  the  EMT  has  an  appropriate  lag 
time  between  treatments. 

It  is  well  known,  well  established,  and  often  repeated  that  “all 
models  are  wrong,  but  some  are  useful.”41  The  previous  pages  have  been  a 
discussion  of  how  the  author  simulated  several  effects  that  capture  the  essence 
of  emergency  first  response  to  a  crisis  event.  There  are  some  limitations  to  this 
model  and  modeling  technique.  The  next  section  discusses  the  limitations 


41George  Box,  “Robustness  in  the  Strategy  of  Scientific  Model  Building,”  in  Robert  Launer 
and  Graham  Wilkinson  (Eds.)  Robustness  in  Statistics,  1979,  p.  202. 
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identified,  the  artificialities  associated  with  this  type  of  simulations,  and  the 
assumptions  made  to  work  around  those  artificialities. 

B.  MODEL  LIMITATIONS  AND  ARTIFICIALITIES 
1.  Model  Limitations 

It  is  important  to  note  that  the  Pythagoras  model  is  currently  a 
beta  version.  The  developers  of  the  model  have  been  particularly  helpful  in 
trying  to  provide  a  quality  model  to  use  for  this  thesis  and  others  that 
are  ongoing. 


a.  Invulnerable  Agents 

As  mentioned  previously  in  the  discussion  of  timestep  selection, 
agents  jump  from  pixel  to  pixel  with  each  timestep.  It  is  possible  for  an  agent  to 
jump  into  a  terrain  feature  from  which  the  agent  cannot  move.  For  example,  if  an 
agent  is  standing  next  to  a  feature  with  movement  factor  0  (e.g.,  a  wall)  and  that 
agent  has  a  desire  to  move  past  the  wall,  that  agent  may  jump  into  the  wall.  The 
agent  can  no  longer  move  because  the  feature’s  movement  factor  is  0. 

This  problem  is  especially  noticeable  in  the  event  that  a  terrorist 
being  tracked  by  a  police  patrolman  jumps  into  a  wall.  The  patrolman  sees  the 
terrorist’s  last  position  and  sees  the  place  at  which  the  terrorist  went  into  the  wall. 
The  patrolman  will  continue  to  engage  the  terrorist,  even  though  he  cannot  see 
him,  due  to  the  wall  having  a  concealment  factor  of  1.0.  Since  the  wall  has  a 
protection  factor  of  1.0,  the  terrorist  cannot  be  killed.  The  result  is  that  the 
patrolman  remains  at  the  wall  until  the  end  of  the  simulation.  In  addition,  the 
patrol  communicates  with  his  unit  members,  relaying  the  enemy’s  location, 
resulting  in  the  entire  unit  being  stationed  at  the  wall,  shooting  ineffectively  at  the 
wall,  for  the  rest  of  the  scenario.  This  problem  does  not  occur  frequently  enough 
to  impact  the  analysis. 
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Use  of  Fuel 


b. 

The  modeler  can  use  the  concept  of  fuel  in  Pythagoras  in  several 
ways,  one  of  which  is  actually  using  it  as  fuel.  One  limitation  of  Pythagoras  is  the 
mechanism  by  which  fuel  is  passed  from  the  supplier  to  the  consumer.  The 
supplier  provides  all  of  the  necessary  fuel  in  one  timestep  and  then  moves  to  the 
next  consumer.  This  situation  is  an  accurate  representation  of  the 
supplier/consumer  relationship  if  the  supplier  is  dropping  off  a  bulk  fuel  package. 
There  is  no  ability  for  a  modeler  to  represent  fuel  being  passed  from  supplier  to 
consumer  over  time,  e.g.,  pumped  at  a  certain  rate  of  units  per  timestep.  In  this 
simulation,  fuel  represented  health.  If  there  was  a  capability  to  simulate  a  rate  of 
fuel  transfer,  the  EMTs  stabilizing  stretcher  wounded  civilians  would  have 
needed  to  stay  directly  with  their  patient  until  sufficient  fuel  was  provided. 

This  deficiency  extends  to  the  use  of  resources  X,  Y,  and  Z,  which 
are  modeled  in  the  same  manner  as  fuel  in  Pythagoras.  The  supplier/consumer 
exchange  deficiency  is  reported  to  have  been  fixed  in  Pythagoras  version  1.10.1, 
but  this  has  not  been  verified  by  the  author. 

c.  Weapon  Usage 

There  are  situations  in  the  current  simulation  in  which  agents  are 
targeted  with  weapons  they  should  not  be  targeted  with.  For  example,  police  are 
supposed  to  target  only  enemies  with  the  9mm  pistol.  Civilians  are  considered 
friends  to  the  police  force.  Unfortunately,  situations  occur  in  which  police  target 
civilians  with  the  9mm  and  kill  them.  This  occurrence  is  (fortunately)  relatively 
rare  and  does  not  provide  a  significant  impact  on  the  Measures  of  Effectiveness 
(MOEs)  of  this  thesis. 

There  are  instances  within  the  simulation  in  which  weapons  that 
are  restorative  actually  kill.  These  isolated  instances  occurred  when  police  used 
their  orders,  a  paintball  restorative  weapon,  to  influence  civilians.  This 
occurrence  is  also  rare  and  does  not  impact  the  analysis  of  MOEs  in 
this  research. 
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d.  Injured  Friend  Identification 

Currently,  agents  do  not  react  appropriately  using  the  move 
“toward  injured  friend”  logic.  In  fact,  this  desire  does  not  appear  to  work  at  all. 

e.  Identification  of  Undetectable  Agents 

Prior  to  timestep  375,  the  terrorist  gunman  is  in  a  state  that  sets  his 
detection  factor  across  all  sensing  bands  to  be  0.0.  Within  the  vignette,  there  are 
several  instances  in  which  the  terrorist  gunman  is  identified  when  he  should  be 
completely  undetectable.  This  model  deficiency  did  not  impact  analysis  because 
as  the  gunman  was  identified,  appropriate  action  took  place,  i.e.,  location  was 
communicated  to  friendly  units,  the  friendly  units  consolidated  and  killed 

the  terrorist(s). 

f.  Error  Reporting 

Error  reporting  in  Pythagoras  is  not  user  friendly.  If  there  is  an  error 
in  the  Pythagoras  Java  code,  a  generic  error  message  appears  on  the  screen 
and  the  scenario  file  will  not  open.  The  next  step  for  the  user  to  follow  is  to  open 
the  .xml  file  in  an  xml  editor  and  look  for  errors.  The  user  is  not  given  an 
indication  where  to  look  in  the  code  for  the  error.  Heuristics  exist  to  guide  the 
user  in  looking  through  the  code,  but  often  fail.  There  are  times  that  a  working 
file  is  compared  to  a  nonworking  file  and  the  differences  should  not  cause  errors 
in  running  the  file.  It  would  be  useful  for  the  developers  of  Pythagoras  to  provide 
an  error  message  that  directs  the  user  to  the  specific  area  of  computer  code  that 
fails  in  a  model  run. 

2.  Artificialities 

a.  Terrain 

(1)  Rubbling.  The  rubbling  in  this  simulation  was  created 
without  the  use  of  scientific  data.  The  author  created  the  rubble  to  capture  the 
effect  of  firefighters  moving  to  the  area  and  searching. 
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(2)  Bomb  Damage.  In  reality,  a  7,300-pound  bomb  would 
have  damaged  more  than  two  buildings.  The  desired  effect  of  this  simulation 
was  to  capture  first  responder  interaction  with  the  civilian  crowd  that  was  close  to 
the  explosion.  This  effect  was  captured  in  a  satisfactory  manner  without  detailed 
research  into  building  structure  thresholds. 

(3)  Triage  Points.  Due  to  the  static  nature  of  the  Pythagoras 
terrain,  triage  points  had  to  be  set  up  before  the  simulation  started.  The 
assumption  that  the  author  made  in  using  prepositioned  triage  points  is  that  the 
location  of  these  points  was  establish  in  pre-event  planning. 

b.  Civilian  Effects 

Not  addressed  in  this  simulation  are  the  unpredictable  effects  of 
crises  on  humans.  Inevitably,  there  are  civilians  who  will  try  to  perform  first  aid 
on  wounded  people,  help  carry  stretchers,  and  seek  the  opportunity  to  help  the 
first  responders.  Also  apparent,  with  the  events  surrounding  the  response  to  the 
2005  hurricanes,  is  that  some  people  will  take  advantage  of  the  situation  by 
becoming  unruly,  or  even  engaging  in  criminal  activity.  Neither  effect  is 
simulated  in  this  vignette. 

This  chapter  provided  a  detailed,  step-by-step  discussion  of  the 
technique  used  to  model  emergency  response  to  a  crisis  in  a  MAS.  The 
modeling  described  here  corresponds  to  steps  8  and  9  of  the  methodology 
outlined  in  Figure  6  and  subsequently  discussed.  The  following  chapter  details 
steps  10  and  11  of  Figure  5,  identifying  factors  of  concern  and  their  levels. 
Chapter  V  will  also  discuss  the  experimental  design  process — the  bridge 
between  implementing  the  model  and  analyzing  the  data. 
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V.  EXPERIMENTAL  DESIGN  METHODOLOGY 


This  chapter  describes  steps  11  and  12  of  the  simulation  methodology 
depicted  in  Figure  6,  outlining  the  design  of  experiments  (DOE)  for  this  research. 
The  DOE  is  the  mechanism  the  analyst  uses  to  vary  input  factors,  conditioning 
the  model  to  answer  questions  of  interest;  it  is  the  bridge  between  model 
development  and  analysis  of  data.  The  author  will  describe  the  three  different 
experimental  designs  chosen  to  collect  the  data  needed  to  analyze  the  TOPOFF 
scenario.  Following  the  discussion  of  the  DOE  and  input  factors,  the  author  will 
describe  the  chosen  MOEs  that  answer  the  questions  posed  at  the  beginning  of 
this  research.  The  chapter  concludes  with  a  brief  discussion  of  tools  and 
techniques  that  support  the  development  of  the  DOE  and  the  analysis  of  the 
resultant  data. 

A.  METHODOLOGY 

The  power  of  simulations,  especially  MAS,  results  from  the  ability  of  an 
analyst  to  quickly  analyze  the  consequences  of  decisions.  In  the  case  of 
emergency  response,  the  decision  may  include  the  mix  of  response  personnel 
and  the  time  at  which  they  are  committed.  The  decision  may  also  be  less 
tangible,  such  as  a  recommendation  on  TTPs  for  a  response  organization. 
Gaining  insight  into  these  questions  requires  testing  the  given  scenario  by 
varying  specific  factors  of  interest  over  a  range  of  values.  This  model,  or  another 
model  that  seeks  to  emulate  human  behavior,  is  stochastic;  i.e.,  it  involves 
variability.  The  stochastic  nature  of  this  model  results  in  the  necessity  of  running 
multiple  replications  of  each  unique  model  setting  to  obtain  an  empirical 
distribution  of  the  range  of  possible  responses. 

The  number  of  possible  variables  in  this  simulation  is  staggering;  it 
highlights  the  need  for  an  efficient  DOE.  There  are  three  “sides”  in  this 
simulation:  civilians,  first  responders,  and  terrorists.  See  Table  16  for  a 
breakdown  of  the  total  number  of  unique  agents  in  this  simulation. 
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Side  Name 

First  Level  Subset 

Number  of 

Total  Number  of 

Sub  Levels 

Unique  Agents 

Civilian 

Walking,  Stretcher  Wounded 

2 

8 

First  Responder 

Police,  Fire,  Medical 

3 

46 

Terrorist 

Agitator,  Gunman 

2 

5 

Table  16.  Agent  Side  Breakdown 


Each  unique  agent  class  has  several  qualities  that  can  be  varied: 

•  Number  of  agents 

•  Vulnerability  of  agents 

•  Agent  marksmanship  (police  and  terrorist) 

•  Agent  communication  effectiveness 

In  addition  to  varying  individual  agent  qualities,  it  is  possible  to  vary  overall 
qualities  associated  with  several  agents  at  once.  For  example: 

•  Effectiveness  of  a  weapon  in  general,  12  weapons  in  the  simulation 
(versus  marksmanship  of  an  individual  agent  using  that  weapon) 

•  Probability  of  successful  communication  of  a  given  piece  of 
equipment,  nine  different  communication  devices  (versus  an 
individual’s  effectiveness  in  using  the  device) 

•  Probability  of  detection  for  the  sensor 

From  above,  the  total  possible  number  of  factors  that  can  be  varied  in  this 


simulation  is  258. 

59  unique  agents  x  4  factors  per  agents  = 

236  factors 

+ 

12  different  communication  devices  = 

1 2  factors 

+ 

9  different  communication  devices  = 

9  factors 

+ 

1  sensor  = 

1  factor 

Of  the  258  possible  factors,  the  author  determined  that  varying  48  factors 
would  provide  the  data  to  answer  the  research  questions  posed.  Using  a  simple 
grid  design,  varying  these  48  factors  at  three  levels  (a  high  level,  medium  level, 
and  low  level)  would  result  in  348  (7.98  x  1022)  unique  model  runs  or  design 
points.  Each  model  run  takes  approximately  40  minutes,  resulting  in  a  need  for 
5.32  x  1022  CPU  hours.  This  simulation  is  a  stochastic  simulation,  meaning  that 
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the  analyst  must  run  several  replications  of  each  design  point  to  achieve 
meaningful  results.  Assuming  that  30  replications  are  sufficient  (many 
simulations  require  50  or  100),  the  entire  simulation  would  require  1.60  x  1024 
CPU  hours,  which  is  about  116  trillion  times  the  approximate  age  of 
the  universe.42 

Given  that  we  do  not  have  that  long  to  wait,  it  is  necessary  to  develop  a 
much  more  efficient  DOE  than  the  traditional  gridded  design,  or  consider  far 
fewer  factors.  The  author  chose  to  find  an  efficient  design.  Robust,  efficient 
designs  of  experiments  enable  analytical  factor  selection,  causing  the  resulting 
analysis  to  be  less  affected  by  personal  judgments  and  bias. 

The  search  for  an  efficient  DOE  quickly  led  to  work  done  by 
U.S.  Army  Lieutenant  Colonel  Thomas  M.  Cioppa,  using  NOLHs.  As  mentioned 
earlier,  data  farming  provides  a  method  to  take  advantage  of  efficient  DOEs, 
such  as  the  NOLH,  and  supercomputers  to  grow  an  abundance  of  data  points  for 
further  exploration.  The  NOLH  design  efficiently  searches  the  high-dimensional 
input  space  defined  by  an  intricate  response  surface.  The  NOLH  has  the 
following  characteristics:43 

•  Approximate  orthogonality  of  all  input  factors 

•  A  collection  of  experimental  cases  representative  of  the  subset  of 

points  in  the  hypercube  of  explanatory  variables  (space  filling) 

•  Ability  to  examine  20  or  more  variables  efficiently 

•  The  flexibility  to  analyze  and  estimate  multiple  effects,  interactions, 
and  thresholds 

•  Requires  minimal  a  priori  assumptions  on  the  response 

•  Easy  design  generation 

•  Ability  to  gracefully  handle  premature  experiment  termination 


42 Age  of  the  Universe.  Retrieved  on  19  May  2006  from  the  World  Wide  Web  at 
http://en.wikipedia.org/wiki/Aqe  of  the  Universe,  using  NASA's  Wilkinson  Microwave  Anisotropy 
Probe  (WMAP). 

43Thomas  M.  Cioppa,  “Efficient  Nearly  Orthogonal  and  Space-Filling  Experimental  Designs 
for  High-Dimensional  Complex  Models,”  Ph.D.  Dissertation,  Operations  Research  Department, 
Naval  Postgraduate  School,  Monterey,  CA,  2002. 
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Refer  to  Cioppa’s  dissertation  for  additional  information  regarding  a 
NOLH.  Cioppa’s  algorithm  requires  only  2,049  design  points  to  analyze  the 
same  solution  space  addressed  by  the  previously  mentioned  gridded  design! 
Instead  of  348  runs,  we  require  2U+1  runs,  saving  orders  of  magnitudes  in 
computing  time.  With  this  design,  it  is  possible  to  execute  the  DOE  with  a 
sufficient  number  of  replications  to  achieve  statistical  relevance,  using  current 
technology,  in  a  reasonable  amount  of  time.  See  Appendix  B  (Design  of 
Experiment  Modeling)  for  detailed  information  about  calculating  the  number  of 
runs  required. 

Cioppa’s  is  an  excellent  design,  but  current  work  ongoing  by 
U.S.  Army  Lieutenant  Colonel  Alejandro  Hernandez  extends  the  ideas  developed 
by  Cioppa.  Hernandez  uses  a  method  incorporating  FRLHs  to  create 
experimental  designs  that  are  even  more  efficient  than  Cioppa’s. 

Hernandez  found  that  using  correlation  reduction  methods,  it  is  possible  to 
create  a  design  that  meets  Cioppa’s  criteria  for  near  orthogonality  when  the 
number  of  design  points  is  greater  than  or  equal  to  49.  In  fact,  the  number  of 
design  points  need  only  be  greater  than  or  equal  to  about  three  times  the  number 
of  variables  to  maintain  Cioppa’s  criteria  of  near  orthogonality,  see  Figure  23. 


n  :=  number  of  design  points  (levels) 
k  :=  number  of  variables 

When  n  >  49, 

n>3k 

Figure  23.  Relationship  Between  the  Number  of  Variables  and  the  Number  of  Levels  in 

Hernandez’s  Design  of  Experiments 

The  result  of  Hernandez’s  work  is  the  requirement  for  only  144  design 
points  to  complete  a  nearly  orthogonal  main  effect  design  of  experiments  that  will 
provide  insight  into  the  behavior  of  this  problem’s  response  surface,  in 
48-dimensional  space.  This  144  x  48  design  represents  a  possible  savings  of 
over  93%  in  terms  of  design  points  and  required  computing  power! 
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The  benefits  that  result  from  using  Hernandez’s  design  come  with  costs 
that  should  be  understood.  This  design  is  less  space  filling  than  designs  using 
Cioppa’s  and  Ye’s44  NOLH  and  OLH  designs.  In  addition,  the  designs  by 
Cioppa  and  Ye  guarantee  orthogonality  or  near  orthogonality  with  the  inclusion  of 
one  interaction  or  one  quadratic  term.  Hernandez  does  not  yet  make  that 
guarantee  with  his  designs. 

B.  EXPERIMENTAL  DESIGNS 

1.  Flexible  Random  Latin  Hypercube  Design  (FRLH) 

The  author  completed  two  separate  experiments  using  the  FRLH  design. 
For  both  experiments,  the  hypercube  generated  by  Hernandez  was  crossed  with 
a  simple  3x1  grid  that  captures  three  different  probabilities  of  communication. 
The  probabilities  of  communication  examined  are:  100%,  75%,  and  50%.  At 
100%,  a  piece  of  information  successfully  passed  between  agents  is  certain  to  be 
received.  At  a  level  of  50%,  the  piece  of  information  is  passed  with  probability 
one-half.  See  Tables  17-19  for  the  factors  varied  in  both  FRLH  experiments. 


Factor  Name 

Low 

Level 

High 

Level 

Desired  Outcome  from  Varying 
Given  Factor 

Number  of  Civilians 
(Integer)* 

81 

244 

Identify  possible  effect  of 
first  responder  to  civilian  ratio 

Civilian  Vulnerability 
(Continuous) 

0 

1 

Identify  effect  of  increased  civilian 
susceptibility  to  terrorist  action 

Desire  of  Civilians  to 

Avoid  Terrorists 
(Continuous) 

0 

1 

Examine  effect  of  civilian  fear 
of  terrorists 

*12  correlated  factors. 


Table  17.  Civilian  Factors  Varied  in  FRLH  Design 


44K.Q.  Ye,  “Orthogonal  Column  Latin  Hypercubes  and  their  Application  in  Computer 
Experiments,”  Journal  of  the  American  Statistical  Association  -  Theories  and  Models,  Vol.  93, 
No.  444,  pp.  1430-1439,  December  1998. 
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Factor  Name 

Low 

Level 

High 

Level 

Desired  Outcome  from  Varying 
Given  Factor 

Number  of  Terrorists, 

Location  1  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to 
police  ratio;  key  terrain  for  terrorists 

Number  of  Terrorists, 

Location  2  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to 
police  ratio;  key  terrain  for  terrorists 

Number  of  Terrorists, 

Location  3  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to 
police  ratio;  key  terrain  for  terrorists 

Number  of  Terrorists, 

Location  4  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to 
police  ratio;  key  terrain  for  terrorists 

Number  of  Terrorist  Gunmen 
(Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to 
police  ratio;  key  terrain  for  terrorists 

Vulnerability  of  Agitators 
(Continuous) 

0 

1 

Identify  effect  of  terrorist  body  armor 

Vulnerability  of  Terrorist 
Gunmen  (Continuous) 

0 

1 

Identify  effect  of  terrorist  body  armor 

Marksmanship  of  Agitators 
(Continuous) 

0 

1 

Capture  effect  of  agitator  effectiveness  in 
inciting  panic 

Marksmanship  of  Terrorist 
Gunmen  (Continuous) 

0 

1 

Capture  effect  of  attriting  first  responder 
forces 

Lethality  of  Bomb  (Integer) 

0 

10 

Identify  possible  effects  of  bomb  strength 

Table  18.  Terrorist  Factors  Varied  in  FRLH  Design 


Factor  Name 

Low 

Level 

High 

Level 

Desired  Outcome  from  Varying 

Given  Factor 

Number  of  EMTs  from 

St.  Johns  Hopkins  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to  police  ratio; 
key  terrain  for  first  responders 

Number  of  EMTs  from 

Mercy  Medical  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to  police  ratio; 
key  terrain  for  first  responders 

Number  of  Patrolmen, 

Patrol  Area  1-5*  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to  police  ratio; 
key  terrain  for  first  responders 

Number  of  Follow-on  Police 
at  15  minutes  (Integer) 

0 

7 

Determine  importance  of  time  in  follow-on 
response 

Number  of  Follow-on  Police 
at  30  minutes  (Integer) 

0 

7 

Determine  importance  of  time  in  follow-on 
response 

Number  of  Follow-on  Police 
at  45  minutes  (Integer) 

0 

7 

Determine  importance  of  time  in  follow-on 
response 

Number  of  Traffic  Police, 

Areas  1-12**  (Integer) 

0 

7 

Identify  possible  effect  of  terrorist  to  police  ratio; 
key  terrain  for  first  responders 

Number  of  Personnel  on 

SWAT  Team  (Integer) 

5 

15 

Examine  impact  of  SWAT  team  in  current 
posture  (waiting  for  call  to  attack) 

Police  Marksmanship 
(Continuous) 

0 

1 

Examine  importance  of  police  effectiveness 
(against  terrorist,  for  civilians) 

*5  separately  altered  factors;  **12  separately  altered  factors. 


Table  19.  First  Responder  Factors  Varied  in  FRLH  Design 
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There  are  two  criteria  that  Cioppa  uses  to  assess  the  orthogonality  or 
near  orthogonality  of  a  design  matrix  comprised  of  each  input  variable  (column) 
and  design  point  (row).  These  two  criteria  are  the  absolute  maximum  pairwise 
correlation  and  the  design  matrix  condition  number.45 

The  absolute  maximum  pairwise  correlation  is  computed  by  calculating  the 
correlation  between  every  pair  of  input  columns  in  the  design  matrix,  then  taking 
the  largest  absolute  value.  A  value  of  0  is  best,  representing  an  orthogonal 
design  matrix.  A  value  of  1  is  worst,  meaning  that  at  least  1  column  is  perfectly 
correlated  with  another  column. 

The  condition  number  of  the  design  matrix  is  used  to  “examine  the 
sensitivities  of  a  linear  system.”  The  condition  number  can  also  reveal  the 
degree  of  nonorthogonality  of  a  given  design  matrix.  The  best  value  possible  for 
condition  number  is  1,  associated  with  an  orthogonal  design  matrix. 
Nonorthogonal  design  matrices  have  condition  numbers  greater  than  1 ,  meaning 
that  the  design  matrix  has  a  degree  of  multicollinearity. 

The  designs  created  by  Hernandez  for  this  research  meet  Cioppa’s  criteria 
for  near  orthogonality.  One  design  consists  of  a  144  design  points,  and  is 
described  in  detail  in  Section  V.B.I.a.  This  design  has  an  absolute  maximum 
pairwise  correlation  of  0.008  and  a  condition  number  of  1.046.  The  second 
design  is  detailed  in  Section  V.B.I.b.  This  design,  consisting  of  1,008  design 
points,  has  an  absolute  maximum  pairwise  correlation  of  0.001  and  a  condition 
number  of  1 .006.46 


a.  Small  FRLH  Experiment 

The  first  experimental  run  fully  capitalized  on  the  efficiency  of 
Hernandez’s  FRLH  design.  This  experiment  involves  the  use  of  Hernandez’s 


45Thomas  M.  Cioppa,  “Efficient  Nearly  Orthogonal  and  Space-Filling  Experimental  Designs 
for  High-Dimensional  Complex  Models,”  Ph.D.  Dissertation,  Operations  Research  Department, 
Naval  Postgraduate  School,  Monterey,  CA,  2002. 

46E-mail  from  LTC  Hernandez  titled  “Roginski  New  Designs,”  11  May  2006,  office 
communication. 
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“base”  design,  using  144  design  points,  allowing  for  the  possible  use  of 
43  numeric  factors  and  5  qualitative  factors  (48  total  factors).  Based  on  a  final 
analysis  of  the  factors  to  be  analyzed,  the  author  used  37  of  the  43  unique 
columns,  and  did  not  use  the  five  qualitative  factors.  The  effect  of  this  decision 
on  the  design’s  maximum  pairwise  correlation  and  condition  number  was  to  drive 
them  lower,  making  the  design  even  more  nearly  orthogonal. 

The  total  number  of  runs  in  the  small  FRLH  experiment  is 
144  x  15x3  (each  level  of  communications  grid),  or  6,480  runs.  The  advantage 
of  this  experiment  is  its  relatively  short  run  time,  when  compared  to  the  design 
discussed  in  the  next  section.  The  author  received  data  back  from  the  MHPCC 
in  three  days,  versus  a  three  week  turn  around  time  for  the  large  experiment. 
The  run  resulted  in  approximately  4,320  rows  and  5,310  columns  of  raw  data  for 
analysis. 

The  disadvantage  of  this  design  is  that  its  sparse  coverage  of  the 
solution  space  results  in  a  much  more  granular  understanding  of  the  response 
surface.  In  situations  which  the  response  surface  is  complex,  important  detail 
may  be  missing  from  the  data. 

b.  Large  FRLH  Experiment 

The  second  experiment  run  was  the  largest,  combining  the 
efficiency  of  Hernandez’s  FRLH  design  with  the  space  filling  nature  of 
Cioppa’s  design.  The  same  37  variables  used  in  the  previous  experiment  are 
used  in  this  experiment;  a  heuristic  approach  was  used  to  settle  on  1 ,008  as  the 
number  of  total  design  points. 

The  advantage  of  using  this  approach  is  the  space  filling  nature  of 
using  over  1,000  design  points.  This  number  of  design  points  enables  much 
better  coverage  of  the  solution  space  than  the  144  design  point  experiment, 
previously  discussed.  Greater  coverage  of  the  overall  solution  space  results  in  a 
better  understanding  of  the  response  surface,  and  less  of  a  chance  to  miss  an 
important  result.  The  disadvantage  of  this  design  is  that,  while  relatively  efficient 
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compared  to  a  gridded  design,  this  experiment  still  takes  longer  to  run  than  the 
smaller  FRLH  design. 

Using  the  output  of  previous  runs,  the  author  determined  that 
15  replications  of  each  design  point  are  sufficient  for  analysis  of  data.  The  total 
number  of  runs  in  the  large  FRLH  experiments  is  1,008  x  15  x  3  (each  level  of 
communications  grid),  or  45,360.  This  simulation  requires  40  minutes  of 
CPU  time  for  each  run;  therefore,  the  expected  length  of  this  experiment  was 
30,240  CPU  hours.  The  set  of  runs  was  completed  with  the  help  of  the  MHPCC 
supercomputers  in  about  three  weeks,  resulting  in  approximately  45,360  rows 
and  5,310  columns  of  raw  data  for  analysis. 

2.  Gridded  Design 

The  author  also  used  a  gridded  design,  again  run  at  the  MHPCC.  The 
intent  of  this  design  is  to  focus  directly  on  a  few  factors,  while  holding  the  rest 
constant.  The  author  considered  the  following  variables  and  levels  in  this  design 
(see  Table  20).  The  total  number  of  runs  in  the  gridded  experiment  is 
53  (three  variables  at  five  levels  each)  x  20  replications,  or  2,500  runs  of  the 
simulation.  The  expected  time  for  2,500  runs  at  MHPCC  was  about  1 ,666  CPU 
hours.  The  author  received  data  back  from  Maui  in  two  days,  versus  a 
three-week  turn  around  time  for  the  large  experiment. 


Factor  Name 

Low 

Level 

High 

Level 

Step 

Number 
of  Levels 

Desired  Outcome  From  Varying 
Given  Factor 

Probability  of 
Communication 

0.1 

0.9 

0.2 

5 

Identify  effect  of  communication 
equipment  on  response 

Number  of 

Traffic  Police* 

0 

5 

1 

5 

Identify  effect  of  ratio  of  traffic 
police  to  terrorists,  and  to  patrols 

Number  of 
Patrolmen** 

0 

5 

1 

5 

Identify  effect  of  ratio  of  patrolmen 
to  terrorists,  and  traffic  cops 

*Each  traffic  location  is  given  the  same  number;  **Each  patrol  location  is  given 
the  same  number. 


Table  20.  Gridded  Design  Completed  at  MHPCC 
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3. 


Measure  of  Effectiveness 


Analyzing  a  data  set  that  is  45,000  rows  by  5,300  columns  or  even 
6,000  rows  by  5,300  columns,  is  a  bewildering  task  if  the  analyst  lacks  focus  and 
scope.  MOEs  provide  the  scope  and  focus  that  the  analyst  needs  to  provide 
timely  support  to  the  decision  maker.  The  analyst  establishes  MOEs  that  help 
provide  insights  and  answers  to  research  questions  posed  at  the  beginning  of  a 
given  project.  To  understand  the  use  of  MOEs,  it  is  important  to  understand  the 
difference  between  MOEs  and  measures  of  performance  (MOPs).  Sections  a 
and  b  below  show  the  definitions  of  a  MOP  and  MOE  from  the  Defense  Modeling 
and  Simulation  Office  (DMSO). 

a.  Measure  of  Performance,  Defined 47 

A  measure  of  how  the  system/individual  performs  its  functions  in  a 
given  environment  (e.g.,  number  of  targets  detected,  reaction  time,  number  of 
targets  nominated,  susceptibility  of  deception,  task  completion  time).  It  is  closely 
related  to  inherent  parameters  (physical  and  structural),  but  measures  attributes 
of  system  behavior. 

b.  Measure  of  Effectiveness,  Defined 18 

A  qualitative  or  quantitative  measure  of  the  performance  of  a  model 
or  simulation  or  a  characteristic  that  indicates  the  degree  to  which  it  performs  the 
task  or  meets  an  operational  objective  or  requirement  under  specified  conditions. 

c.  Use  of  MOEs 

MOPs  determine  how  well  one  small  entity  fared  in  the  mission. 
MOEs  identify  how  well  the  mission  was  accomplished  overall.49  Establishing 

47DMSO,  definition  of  Measure  of  Performance.  Retrieved  21  on  May  2006  from  the  World 
Wide  Web  at  https://www.dmso.mil/public/resources/qlossary/results?do=qet&search  text=MOP 

48DMSO,  definition  of  Measure  of  Effectiveness.  Retrieved  on  21  May  2006  from  the  World 
Wide  Web  at  https://www.dmso.mil/public/resources/qlossary/results?do=qet&search  text=MOE 

49Donald  E.  Brown,  Ph.D.,  and  C.  Donald  Robinson,  “Development  of  Metrics  to  Evaluate 
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defensible  MOEs  is  much  more  of  a  challenge  than  establishing  interesting 
MOPs,  especially  in  an  area  that  may  directly  impact  peoples’  lives,  such  as 
emergency  response. 

For  example,  baseball  fans  compare  their  favorite  players’  batting 
average,  on  base  percentage,  and  number  of  runs  batted,  to  try  to  determine  the 
better  player.  These  are  measures  of  the  players’  performance,  not 
effectiveness.  To  measure  a  player’s  true  effectiveness,  we  must  ascertain  how 
much  that  player  contributes  to  winning  the  game,  winning  the  division,  or  the 
World  Series.50 

As  stated  previously,  MOEs  are  established  to  answer  questions 
posed  at  the  beginning  of  research.  Below  are  the  four  MOEs  selected  for  this 
research,  followed  by  Table  21  that  highlights  the  research  questions  associated 
with  each  MOE.  The  MOEs  address  the  effectiveness  of  the  first  response 
agencies  in  mitigating  the  effects  of  the  crisis  event,  seeking  to  achieve  the 
following  goals: 

•  Stabilize  wounded  civilians 

•  Restore  calm  in  the  area 

•  Eliminate  further  threats 

•  Maintain  safety  of  first  responders 

MOEs 

•  MOE  1.  The  first  MOE  is  the  percentage  of  civilians  that  have  not 

received  sufficient  medical  treatment  by  the  end  of  the  first  hour. 

This  MOE  directly  addresses  the  first  priority  established  above  by 


Effectiveness  of  Emergency  Response  Operations,”  presented  at  the  10th  International  Command 
and  Control  Research  and  Technology  Symposium,  The  Future  of  C2,  June  2005.  Retrieved  on 
2  December  2005  from  the  World  Wide  Web  at  www.dodccrp.org/events/2005/10th/papers/ 
326.pdf 

50Brian  G.  McCue,  Christine  A.  Hughes,  and  Kathleen  M.  Ward,  “Analysis  Planning  for 
Domestic  Weapon-of-Mass-Destruction  Exercise,”  The  CNA  Corporation,  The  Occasional  Paper 
Series,  (IPR)  10856,  May  2003,  pp.  21-22.  Retrieved  on  2  December  2005  from  the  World  Wide 
Web  at  www.cna.org/documents/IPR1 0856  1  .pdf 
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measuring  the  effectiveness  of  first  responders  in  the 
“Golden  Hour,”  or  the  first  hour  after  the  bomb  blast. 

•  MOE  2.  The  second  MOE  is  similar  to  the  first  in  that  it  measures 
the  effectiveness  of  the  first  responders.  The  second  MOE  is  the 
percentage  of  civilians  at  the  triage  points  at  the  end  of  the 
first  hour.  Civilians  only  go  to  the  triage  points  after  first  responders 
have  successfully  interacted  with  them,  provided  aid,  and  assuaged 
their  feeling  of  panic.  This  MOE  highlights  the  success  of 
first  responders  in  achieving  the  second  priority  outlined  above.  It 
measures  the  first  response  force’s  psychological  effectiveness  on 
the  civilians,  where  the  first  MOE  measures  physical  effectiveness. 

•  MOE  3.  The  third  MOE  is  the  means  that  the  analyst  will  use  to 
determine  success  in  achieving  the  third  priority.  The  third  MOE  is 
the  percentage  of  times  among  the  replications  for  a  given  design 
point  that  all  terrorist  gunmen  were  neutralized.  Total  neutralization 
of  the  gunmen  means  there  is  no  further  lethal  threat. 

•  MOE  4.  The  fourth  MOE  is  the  percentage  of  first  responders 
injured  or  killed  during  response  operations  in  the  first  hour.  Using 
this  MOE,  the  analyst  can  gain  insight  into  the  factors  that  are  most 
important  in  determining  survivability  of  first  responders. 


Research  Question 

MOE 

What  is  an  appropriate  methodology  for  use  of  a  MAS 
environment  in  the  modeling  of  emergency  response  to  the 
simulated  VBIED  that  explodes  near  the  amphitheatre  in 

Baltimore’s  Inner  Harbor? 

Figure  6, 
Step  10 

What  is  the  most  appropriate  mix  of  police,  fire,  and 
medical  assets? 

1,2 

What  is  the  most  effective  interagency  communication  architecture 
for  emergency  response  to  VBIED  in  Baltimore’s  Inner  Harbor? 

3,4 

Table  21.  Research  Questions  and  Associated  MOEs 


C.  EXPERIMENTAL  DESIGN  TOOLS  AND  TECHNIQUES 

Visual  observation  of  the  Pythagoras  model  provides  a  certain  degree  of 
value;  however,  the  purpose  of  a  MAS,  such  as  Pythagoras,  is  to  view  a 
problem’s  high-dimensional  space.  People  can  see  in  three  dimensions.  With  a 
degree  of  chart  wizardry,  one  can  argue  that  it  is  possible  to  gain  an  idea  of 
four-or  five-dimensional  space.  The  response  surface  associated  with  this 
problem  is  in  40+  dimensional  space,  requiring  the  use  of  several  different  tools. 
This  section  describes  those  tools  used  in  this  research. 
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The  tools  bridging  Pythagoras-produced  data  to  the  analysis  conducted 
include  spreadsheet  modeling  with  Excel,  Tiller®,  and  XML.  As  described  in 
Chapter  IV  (Model  Development)  the  author  maintains  that  spreadsheet 
modeling  provides  an  organized  method  to  perform  the  thought  process,  while 
simultaneously  cataloging  important  modeling  parameters. 

1.  Spreadsheet  Modeling  with  Excel 

Appendix  B,  Section  B,  “DOE  Spreadsheet  Modeling,”  outlines  the 
crossed  FRLH  DOEs,  both  large  (1,008  design  points)  and  small 
(144  design  points).  There  are  six  spreadsheets. 

1.  The  first  is  the  factor  description  and  is  similar  to  that  of 

Tables  17-19.  It  outlines  both  the  decision51  (associated  with  first 
responders)  and  noise52  (associated  with  civilians  and  terrorists) 
factors  creating  the  robust  design. 

2.  The  second  spreadsheet  is  a  FRLH  coded  spreadsheet  for 

40  factors  detailing  the  factor  levels  used  at  each  of  the 
1,008  design  points  in  the  large  FRLH  experiment.53 

3.  The  third  spreadsheet  is  a  design  file  and  looks  very  similar  to  the 
second.  This  file  adds  an  additional  11  correlated  factors.  These 
are  correlated  to  the  civilian  instance  factors.  The  correlation 
represents  an  assumption  that  the  author  made  in  scaling  all 
different  classes  of  civilians  to  maintain  the  same  proportionality  as 
in  the  EPiCS  simulation  conducted  by  TRAC-WSMR.  The  design 
file  incorporates  the  final  crossed  NOLH  DOE  with 
3,024  design  points. 

4.  The  fourth  spreadsheet  is  a  FRLH-coded  spreadsheet  for 

40  factors  detailing  the  factor  levels  used  at  each  of  the  144  design 
points  in  the  small  FRLH  experiment.54 


51  Decision  Variable:  Variable  that  can  be  controlled  by  a  decision  maker  for  first-response 
forces. 

52Noise  Variable:  Variable  that  cannot  be  affected  by  first  response  decision  makers. 

53E-mail  from  LTC  Alejandro  Hernandez  titled  “Roginski  Designs,”  sent  11  May  2006,  office 
communication.  FRLH  40  Factors,  coded  by  LTC  Alejandro  Hernandez,  Naval  Postgraduate 
School,  Monterey,  CA. 

54 1  bid. 
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5.  The  fifth  spreadsheet  is  a  design  file  and  looks  very  similar  to  the 
second.  This  file  adds  an  additional  11  correlated  factors.  These 
are  correlated  to  the  civilian  instance  factors.  The  correlation 
represents  an  assumption  that  the  author  made  in  scaling  all 
different  classes  of  civilians  to  maintain  the  same  proportionality  as 
in  the  EPiCS  simulation  conducted  by  TRAC-WSMR.  The  design 
file  incorporates  the  final  crossed  NOLH  DOE  with 
432  design  points. 

6.  The  sixth  spreadsheet  results  from  Visual  Basic  code  written  by 
Major  Chris  Michel,  USMC,  that  takes  row  data  from  the  FRLH 
spreadsheets  and  put  it  in  a  format  to  be  used  later  in  the  Tiller®. 
The  author  will  describe  this  code  in  Section  V.C.1 . 

2.  Extensible  Markup  Language  (XML) 

Though  Pythagoras  offers  an  easily  viewed  GUI  to  input  data  values, 
analysts  may  also  build  Pythagoras  scenarios  and  edit  them  using  the  XML,  as 
all  Pythagoras  databases  are  stored  and  transmitted  in  XML.  XML  offers  a 
simple  and  very  flexible  text  format  device  derived  from  SGML 
(ISO  8879).  Technicians  originally  designed  SGML  to  meet  the  challenges  of 
large-scale  electronic  publishing;  XML  also  plays  an  increasingly  important  role  in 
the  exchange  of  a  wide  variety  of  data  on  the  Internet.55  Storing  scenarios  in 
XML  permits  the  analyst  to  transmit  scenario  files  quite  rapidly  over  the  Internet 
to  perform  data  farming  techniques.  This  process  occurs  with  agencies  such  as 
the  MHPCC  and  enables  thousands  of  design  points  to  run  over  a  networked 
cluster  of  computers  in  a  short  amount  of  time.56 

3.  Tiller® 

The  Tiller,  Version  0.7. 0.0,  Copyright  2004  Referentia  Systems 
Incorporated,  is  a  product  developed  in  support  of  Project  Albert  and  the 
Marine  Corps  Warfighting  Laboratory.  Its  primary  purpose  is  to  prepare  model 


55W3C,  Extensible  Markup  Language.  Retrieved  on  21  May  2006  from  the  World  Wide  Web 
at  http://www.w3.orq/XML 

56Charles  A.  Sulewski,  “An  Exploration  of  Unmanned  Aerial  Vehicles  in  the  Army’s  Future 
Combat  Systems  Family  of  Systems,”  Masters  Thesis,  Naval  Postgraduate  School,  Monterey, 
CA,  December  2005. 
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XML  scenarios  for  data  farming.  The  Tiller®  outputs  two  files  the  user  must 
submit  together  to  execute  a  data  farming  job. 

The  first  file  is  basecase.xml,  which  is  the  model  scenario  file.  For 
Pythagoras  models,  the  only  change  to  the  scenario  file  is  in  changing  the  name 
to  basecase.  The  author  identified  that  it  can  be  more  efficient  to  manually 
change  the  name  of  the  scenario  file,  rather  than  use  the  file  generated  by  the 
Tiller®.  The  file  generated  by  the  Tiller®  can  be  up  to  33%  larger  than  the  base 
scenario  file,  with  no  added  functionality.  The  second  file  generated  by  the 
Tiller®  is  a  usable  study.xml  file  containing  the  chosen  DOE  for  running  at 
the  MHPCC. 

The  Tiller®  application  may  be  used  alone  to  process  the  DOE.  The 
author  used  the  Tiller’s®  study.xml  file  as  a  starting  point  from  which  to  “lockstep” 
several  values  together.  Lockstepping  is  the  process  by  which  several  factors 
are  changed,  but  some  values  are  changed  together.  The  result  of  submitting 
the  altered  study  file  was  that  the  author  changed  104  variables  in  each  design 
point,  using  37  different  factors  from  Hernandez’s  FRLH. 

4.  Excel  Interface  by  Michel 

Major  Christopher  Michel,  USMC,  developed  Visual  Basic  code  that 
provides  a  bridge  between  the  FRLH  developed  by  Hernandez  and  the  Tiller® 
developed  by  Referentia.  The  user  copies  XML  code  that  defines  each 
excursion  to  be  run.  Michel’s  code  causes  Excel  to  snap  to  the  worksheet  with 
the  FRLH  and  copy  a  row  of  factors.  Next,  the  program  snaps  to  the  worksheet 
that  the  user  has  prepared  to  receive  the  design  points  and  transposes  the 
factors  into  the  appropriate  cells.  This  process  is  repeated  until  the  final  row  of 
the  FRLH  is  placed  into  the  spreadsheet.  All  that  remains  for  the  user  to  do  is 
copy  the  first  three  columns  of  the  worksheet  and  paste  those  cells  into  the 


95 


study.xml  file.57  Refer  to  Appendix  B,  Section  B  (Excel  Macro  by  Michel)  to  see 
the  Visual  Basic  code. 

This  chapter  outlined  the  procedures  for  completing  steps  1 1  and  12  of  the 
simulation  methodology  discussed  in  Figure  5,  which  discusses  the  efficient 
experimental  design  used  in  this  research.  The  following  chapter  will 
demonstrate  the  accomplishment  of  step  13,  the  analysis  of  the  data  resultant 
from  implementing  the  DOE. 


57Christopher  Michel,  “Supporting  A  Marine  Air  Ground  Task  Force  With  Appropriate 
Quantities  of  Ground  Based  Fire  Support,”  Masters  Thesis,  Naval  Postgraduate  School, 
Monterey,  CA,  September  2006. 
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VI.  DATA  ANALYSIS 


This  chapter  describes  step  13  of  the  simulation  methodology  depicted  in 
Figure  6,  the  analysis  of  data  developed  from  executing  the  experimental  design 
discussed  in  the  previous  chapter.  It  also  addresses  the  tools  and  techniques 
used  to  collect  and  “clean”  the  data.  Next,  the  thesis-based  observations  provide 
the  analysis  that  answers  our  research  questions.  The  insights  into  these 
research  questions  are  summarized  in  the  conclusions  and  recommendations  for 
future  study  that  comprise  Chapter  VII. 

A.  DATA  COLLECTION  AND  CLEANING 

1.  Clementine 

Clementine  is  a  data  mining  software  package  that  contains  features  used 
throughout  most  of  the  data  cleaning  process.  Data  cleaning  is  the  process  by 
which  raw  data  from  an  experiment  are  transformed  into  a  format  and  layout  that 
is  conducive  to  conducting  analysis.  To  facilitate  this  process,  and  other  data 
mining  processes,  Clementine  provides  a  GUI-based  suite  that  enables  quick 
and  easy  manipulation  of  the  initial  data  sets  into  manageable  sets  for  analysis. 

The  feature  of  Clementine  that  was  most  important  in  data  cleaning  was 
the  development  of  a  data  “stream”  on  the  background  palette.58  A  Clementine 
stream  is  an  intuitive,  graphical  representation  of  a  process  on  the  computer 
screen.  For  example,  this  data  cleaning  process  consisted  of  importing  a  data 
file,  removing  columns,  changing  column  names  to  make  data  analysis  more 
intuitive,  and  exporting  the  transformed  data  into  another  file.  To  repeat  the 
process  for  another  file,  all  that  need  happen  is  for  the  new  file  to  be  uploaded 
onto  the  palette,  and  then  the  stream  attached  to  the  new  file.  The  new  stream  is 
executed  and  immediately  cleaned.  See  Appendix  C  (Data  Analysis)  for  an 
illustration  of  a  Clementine  stream  used  in  this  data  cleaning  process. 

58Clementine  product  description.  Retrieved  on  30  May  2006  from  the  World  Wide  Web  at 
http://www.spss.com/clementine 
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After  all  files  from  a  given  experiment  were  ready,  the  author  imported  the 
new  files  into  Excel  and  inserted  a  column  that  identified  the  associated 
probability  of  communication  for  the  given  file.  The  new  file  was  reloaded  to  the 
Clementine  palette  and  appended  to  the  other  files  that  resulted  from  the 
experiment.  This  consolidated  file  was  then  exported  into  a  file  format  to  be 
analyzed  with  JMP  Statistical  Discovery  Software™.  The  next  section  provides 
more  information  about  JMP. 

Clementine  facilitated  the  data  cleaning  process  for  ten  total  files,  in  three 
separate  experiments,  resulting  in  the  removal  of  nearly  52,000  columns  of  data 
from  the  files  to  be  analyzed.  In  Clementine,  this  process  took  approximately 
three  hours.  Based  on  preliminary  work  in  SPLUS,  executing  the  same  process 
in  SPLUS  would  have  taken  twice  as  long,  even  with  its  new  Big  Data  library 
feature.  In  JMP,  the  process  would  have  consisted  of  52,000  point  and  click 
deletions,  taking  much  longer  than  even  SPLUS. 

2.  Analysis  Software  Tools  (JMP  Statistical  Discovery  Software  ™) 

JMP  Statistical  Discovery  Software™  contains  the  software  features  used 
for  both  the  data  cleaning  and  the  data  analysis  portion  of  this  research.  The 
JMP  statistics  package  is  the  tool  chosen  to  support  the  majority  of  the  data 
analysis  because  it  is  a  useful  statistical  analysis  tool  with  powerful  data 
visualization  features.  JMP  excels  at  helping  analysts  uncover  relationships  and 
outliers  within  the  data.  The  exploration  of  these  data  points  can  lead  to  valuable 
discoveries  and  possible  surprises  in  the  data,  thus  supporting 
better  decision-making.59 

3.  Analysis  Techniques 

Most  large  databases  yield  the  flexibility  to  perform  a  wide  array  of  data 
analysis  techniques.  Though  this  analysis  applies  statistical  tests,  the  core 


c;q 

JMP,  The  Statistical  Discovery  Software.  Retrieved  on  20  April  2006  from  the  World  Wide  Web 
at  http://www.imp.com/product/jmp5  brochure.pdf 
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analysis  focuses  primarily  on  two  techniques:  Classification  and  Regression 
Trees,  and  Multiple  Regression. 

a.  Classification  and  Regression  Trees  (CART) 

The  CART  (Classification  and  Regression  Trees)  algorithm  is  a 
widely  used  statistical  procedure  for  producing  classification  and  regression 
models  with  a  tree-based  structure.  Tree  models  assist  the  analyst  in  the 
identification  of  significant  factors,  by  grouping  similar  points  into  cells.  This 
process  is  recursive;  it  repeats  as  many  times  as  necessary  so  that  each  end 
branch  defines  a  separate  node.60  61  The  regression  tree  yields  a  continuous 
output;  however,  classification  trees  are  discrete  in  nature.62  The  CART 
algorithm  will  classify  significant  response  factors  into  classes  complemented  by 
further  regression  analysis.63 

Throughout  this  analysis,  the  author  used  a  5%  heuristic  to 
determine  when  to  stop  splitting  the  tree.  As  additional  splits  are  made  to  the 
tree,  the  R2  value  is  logged.  Each  R2  value  is  compared  to  the  previous  value, 
when  the  change  in  R2  is  less  than  5%,  no  further  splits  are  made.  The  final  split 
is  then  pruned,  leaving  only  splits  in  the  tree  that  make  an  impact  on  the  R2  of 
greater  than  5%. 

b.  Multiple  Regression 

A  general  regression  analysis  is  a  statistical  process  that 
investigates  the  relationship  between  two  or  more  variables  (factors)  related  in  a 
nondeterministic  fashion.  The  objective  in  multiple  regression  is  to  build  a 


60Douglas  Montgomery,  Elizabeth  Peck,  and  Geoffrey  Vining,  Introduction  to  Linear 
Regression  Analysis,  Third  Edition,  (John  Wiley  and  Sons,  Inc.,  2001),  p.  516. 

61David  Hand,  Heikki  Mannila,  and  Padhraic  Smyth,  Principles  of  Data  Mining,  (MIT  Press, 
Cambridge,  MA,  2001),  p.  145,  343. 

62Hand,  p.  147. 

63lbid  47. 


99 


probabilistic  model  that  relates  a  dependent  variable  y  to  one  or  more 
independent  or  predictor  variables.  The  actual  y  values  in  a  sample  differ  from 
the  predicted  values.  The  errors  or  residuals,  denoted  by  e,  are  the  differences 
between  the  observed  and  predicted  values,  hopefully  possessing  a  normal 
distribution  with  constant  variances.64  Regression  analysis  is  practical  for  gaining 
insight  on  which  predictor  variables  (design  factors)  have  the  greatest 
significance  toward  the  success  of  the  first  response  mission,  as  measured  by 
the  previously  mentioned  MOEs.  Regression  analysis  is  also  useful  in  identifying 
interactions  between  input  variables.65 

B.  RESEARCH  QUESTION-BASED  ANALYSIS 

Each  data  set  developed  through  the  experimental  design  process 
contributed  to  the  analysis  described  in  this  section.  Because  of  data  output 
difficulties  at  MHPCC,  only  two-thirds  of  the  data  from  the  small  FRLH 
experiment  was  available  for  analysis,  corresponding  to  the  144  design  point 
base  design,  with  probability  of  communication  set  at  1.0  and  0.75. 

Before  conducting  analysis  of  the  data  using  trees  and  regression  models, 
the  data  were  grouped  by  like  design  points.  This  model  is  stochastic  and,  as  a 
result,  each  design  point  is  replicated  15  times.  Design  points  are  grouped  and 
desirable  statistics  taken  (e.g.,  means  of  desired  end  of  run  MOEs). 

For  example,  the  raw  data  set  for  the  small  FRLH  experiment  resulted  in 
4,320  rows.  The  first  MOE  addresses  the  percentage  of  civilians  injured  after  the 
end  of  the  first  hour.  The  data  set  is  grouped  by  design  point,  and  the  mean 
number  of  civilians  injured  at  the  end  of  the  first  hour  taken.  The  result  of  this 
approach  is  that  the  actual  data  set  analyzed  for  a  given  problem  is  more 
manageable  than  when  in  its  raw  form.  A  data  that  is  4,320  rows  by 
170  columns  (after  cleaning)  may  be  reduced  to  data  set  of  288  rows  by 
39  columns. 

64Devore,  p.  587. 

65lbid  47. 
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1 .  MOE  1 .  Percentage  of  Civilians  Injured  After  First  Hour 

a.  Flexible  Random  Latin  Hypercube  (FRLH)  Experiment 

The  first  step  of  this  analysis  was  to  develop  a  classification  and 
regression  tree  to  identify  which  variables  have  the  greatest  significance  in 
determining  the  percentage  of  civilians  injured.  Figure  24  shows  the  increase  in 
R2  (a  measure  of  the  amount  of  variance  in  the  data  that  is  explained  by  the 
given  model)  with  the  number  of  splits  in  the  regression  tree,  using  data 
generated  with  the  small  FRLH  model.  Note  that  after  the  third  split  in  the  tree, 
there  is  only  a  small  difference  in  R2.  The  increase  resulting  from  making  four 
splits,  versus  three,  is  only  0.03,  from  0.816  to  0.846,  resulting  in  a  well-fit 
regression  tree. 


R  squared  By  Split 


— •—  Rsquared 


Figure  24.  Analysis  of  R2  by  Regression  Tree  Split  (Small  FRLH  Experiment, 
Response  =  Percentage  of  Injured  Civilians) 

Figure  25  is  the  regression  tree  developed  in  JMP  to  help  gain 
insight  into  the  factors  that  influence  the  percentage  of  civilians  that  remain 
injured  after  the  first  hour.  As  mentioned  previously,  regression  trees  involve  the 
use  of  an  algorithm  that  groups  like  observations  together,  maintaining  the 
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highest  possible  “purity.”  After  the  completion  of  each  step,  the  algorithm 
calculates  the  decrease  in  total  sum  of  squares  resulting  from  the  introduction  of 
each  “candidate  node.”  See  Appendix  C  (Data  Analysis)  for  a  more 
detailed  description. 


Regression  Tree  to  Determine  Percentage  of  Civilians 
Injured  After  the  First  Hour  (R2  =  0.82) 


Figure  25.  Regression  Tree  to  Determine  the  Percentage  of  Civilians  Injured  After  the 

First  Hour  (Small  FRLH  Experiment) 

A  strength  of  regression  trees  is  their  interpretability  and  ease  of 
understanding.  For  example,  see  item  one  on  Figure  25.  Item  one  is  the  first 
split  of  the  regression  tree.  JMP  determined  that  the  division  in  the  data  that 
would  result  in  the  greatest  increase  in  purity  would  be  to  split  the  data  using  the 
variable  “Police  Effectiveness  in  Issuing  Orders  to  Civilians.”  In  effect,  this  is  the 
most  important  single  variable  in  determining  the  number  of  injured  civilians  at 
the  end  of  the  first  hour.  The  increase  in  purity  corresponds  to  the  greatest 
reduction  in  total  sum  of  squares.  The  tree  is  used  as  follows. 

•  There  are  64  design  points  that  correspond  to  a  police 
effectiveness  score  of  less  than  23%.  For  these  cases,  an  average 
of  about  15%  of  the  civilians  remain  injured  at  the  end  of  the  first 
hour  (see  the  boxed  entry  in  the  first  split  of  the  tree). 
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•  There  are  224  design  points  that  have  a  police  effectiveness  score 
of  greater  than  or  equal  to  23%.  For  these  cases,  an  average  of 
approximately  5%  of  the  civilians  remains  injured  after  one  hour. 

Item  2  shows  the  second  split  in  the  tree,  corresponding  to  the 

greatest  reduction  in  total  sum  or  squares  for  the  observations  in  the  node 

(Pol_Effect<.23).  Notice  that  the  worst  case  scenario  in  the  tree  results  from  this 

split.  If  the  police  effectiveness  is  less  than  6%,  we  would  expect  to  see  an 

average  of  about  20%  of  the  civilians  that  remain  at  the  end  of  the  first  hour. 

Item  3  shows  the  third  split  in  the  tree,  corresponding  to  the  third 

largest  reduction  in  total  sum  or  squares  for  the  model.  Notice  that  the  best  case 

scenario  in  the  tree  results  from  this  split.  If  police  effectiveness  is  less  than 

23%,  and  greater  than  43%,  we  would  expect  to  see  only  about  approximately 

4%  of  the  civilians  injured  at  the  end  of  the  first  hour. 

It  is  interesting  that,  after  considering  all  37  independent  variables, 

the  most  important  variable  in  determining  the  number  of  injured  civilians  is  a 

variable  that  the  decision  maker  can  control;  it  is  a  decision  variable.  However, 

the  variable  that  has  the  second  most  impact  is  a  variable  which  the 

decision  maker  cannot  control,  a  noise  variable — the  desire  of  civilians  to  flee 

from  the  enemy.  This  variable  is  not  displayed  on  the  tree;  it  is  the  second 

variable  to  appear  if  the  tree  is  expanded.  This  desire  may  correspond  to  a 

feeling  of  terror  in  identifying  the  enemy  that  is  greater  than  the  comfort  offered 

by  a  first  responder.  The  injured  civilian  runs  away  and  the  first  responder  is  not 

able  to  help. 

It  is  helpful  to  understand  that  there  is  a  certain  degree  of 
randomness  that  prevails  in  our  world.  A  decision  maker  will  mostly  likely  not  be 
able  to  remove  all  uncertainty  from  a  crisis  response  plan.  It  is  helpful  to 
understand  the  relationship  of  controllable  factors  to  uncontrollable  factors.  The 
contour  plot  in  Figure  26  helps  demonstrate  the  relationship  of  our  decision  factor 
(police  effectiveness)  to  the  noise  factor  (desire  to  flee). 
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Contour  Plot  for  Percentjnjured 
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Figure  26.  Relationship  of  Police  Effectiveness  in  Giving  Orders  to  Number  of  Civilians  in 
Determining  Percentage  of  Injured  Civilians  (Small  FRLH  Experiment) 

(best  viewed  in  color) 

This  picture  helps  to  illustrate  the  robust  nature  and  importance  of 
police  effectiveness  in  giving  orders  to  the  civilians  in  the  crowd.  Note  that  when 
police  effectiveness  is  at  just  60%,  the  number  of  injured  civilians  does  not 
increase  appreciably,  even  at  the  highest  levels  of  civilian  desire  to  run  from  the 
enemy.  It  is  only  when  effectiveness  drops  below  20%-30%  that  the  number  of 
civilians  injured  begins  to  increase  with  a  greater  civilian  desire  to  flee.  Note  that 
the  first  split  of  the  regression  tree  is  when  the  police  order  effectiveness 
was  23%. 


Breakpoint: 
Civilians  Running 
From  Enemy:  12 


Breakpoint: 

Police  order 

Effectiveness:  0.23,  0.06 


b.  Gridded  Experiment 

The  analysis  shows  that  the  effectiveness  in  police  issuing  orders 
to  civilians  is  a  dominating  factor  in  determining  the  percentage  of  civilians 
injured  at  the  end  of  the  first  hour  of  response.  There  are  other  questions  that 
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must  be  analyzed  to  gain  an  understanding  of  what  is  most  important  in 
emergency  response. 

•  What  role  does  communication  degradation  play  in 
emergency  response? 

•  Is  there  key  terrain  that  the  emergency  response  teams  must 
ensure  is  secured  to  facilitate  response? 

The  technique  chosen  to  gain  insight  into  those  questions  was  to 
run  a  gridded  experiment  in  which  just  three  variables  are  changed:  probability 
of  communication,  number  of  patrolmen  at  each  location,  and  number  of  traffic 
police  at  each  intersection.  Probability  of  communication  did  not  enter  into  the 
tree  as  a  significant  variable  in  the  small  FRLH  experiment.  In  reality, 
communication  is  very  important  in  emergency  response,  and  one  of  the  points 
highlighted  first  in  most  interagency  AARs.  The  author  was  only  able  to  vary 
communication  over  two  levels  in  the  small  FRLH  experiment.  The  gridded 
experiment  resulted  from  varying  communication  and  both  other  variables  at  five 
levels  each. 

Figure  27  shows  the  increase  in  R2,  with  the  number  of  splits  in  the 
regression  tree,  using  data  generated  with  the  gridded  model.  Note  that 
diminishing  returns  in  R2  occurs  after  the  fifth  split.  The  increase  resulting  from 
making  five  splits,  versus  four,  is  only  0.025,  from  0.642  to  0.667.  The  result  is  a 
tree  that  has  an  R2  of  only  0.642,  lower  than  we  would  like  for  predictive 
purposes;  however,  the  purpose  of  this  tree  is  not  necessarily  prediction,  but 
gaining  insight  into  the  relationship  of  factors  that  the  decision  can  control. 
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Figure  27.  Analysis  of  R2  by  Regression  Tree  Split  (Gridded  Experiment, 
Response  =  Percentage  of  Injured  Civilians) 

Figure  28  is  the  regression  tree  developed  in  JMP,  from  the  gridded 
data  set,  to  help  gain  insight  into  the  factors  that  influence  the  number  of  civilians 
that  remain  injured  after  the  first  hour.  Within  the  tree,  there  is  very  little 
variability  between  the  tree’s  best  (approximately  91%)  and  worst  case  scenario 
(approximately  87%).  This  small  amount  of  variability  means  that  it  may  be 
difficult  to  fit  a  good  model  to  the  data. 
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Figure  28.  Regression  Tree  to  Determine  the  Percentage  of  Civilians  Injured  After  the 

First  Hour  (Gridded  Experiment) 


After  making  five  splits,  it  is  evident  that  communication  does  not 
seem  to  have  significant  importance  in  this  model’s  emulation  of  first  response. 
In  fact,  it  is  not  until  the  tenth  split  (not  shown)  that  the  communication  factor  is 
included  in  the  tree. 

In  retrospect,  the  lack  of  communication  importance  in  this  model 
vignette  is  not  terribly  surprising.  Communication  is  meant  to  serve  many 
functions  in  response,  including,  but  not  limited  to:  passing  situational 
awareness  in  the  form  of  information,  sending  information  about  enemy  locations 
and  dispositions,  and  providing  direction  in  terms  of  orders. 

As  previously  stated,  in  Section  IV.A.5.,  the  actions  simulated  in 
this  research  are  the  adaptation  of  work  that  occurs  by  standing  operation 
procedure  (SOP).  SOPs  are  meant  to  minimize  the  need  for  communication  and 
streamline  operations;  therefore,  it  is  intuitive  that  in  certain  cases, 
communication  factors  may  be  dominated  by  others,  such  as  location  and 
numbers  of  forces,  in  the  early  stages  of  an  emergency.  However,  a  model  could 
easily  be  adjusted  to  account  for  lack  of  SOPs,  undisciplined  following  of  SOPs, 
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or  SOPs  that  cannot  be  followed  because  of  the  situation.  In  these  cases, 
analysis  may  reveal  a  higher  level  of  importance  for  communication  factors. 

2.  MOE  2.  Percentage  of  Civilians  at  the  Medical  Triage  Point 
After  One  Hour 

Analysis  of  the  second  MOE  follows  the  same  pattern  as  the  first. 
Figure  29  shows  the  increase  in  R2  with  the  number  of  splits  in  the  regression 
tree.  Note  that  after  the  second  split  in  the  tree,  the  increase  in  R2  is  small.  The 
increase  resulting  from  making  three  splits,  versus  two,  is  only  0.042,  from 
0.791  to  0.833. 


R  squared  By  Split 


— • —  Rsquared 


Figure  29.  Analysis  of  R2  by  Regression  Tree  Split  (Small  FRLH  Experiment, 
Response  =  Civilians  at  Triage  Site) 

Figure  30  shows  the  regression  tree  that  results  from  splitting  twice. 
Notice  that  police  effectiveness  in  giving  orders  is  the  most  important  variable;  in 
fact,  it  is  the  only  variable  involved  in  the  tree  for  the  first  four  splits. 
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Figure  30.  Regression  Tree  to  Determine  the  Percentage  of  Civilians  at  the  Medical  Station 

(Small  FRLH  Experiment) 

In  the  regression  tree,  police  effectiveness  has  a  dramatic  impact  on  the 
percentage  of  civilians  at  the  triage  point,  accounting  for  nearly  88%  of  the 
variance  after  four  splits.  Figure  31  shows  multiple  regression  output  that  serves 
to  verify  the  relationship  of  police  effectiveness  to  percentage  of  civilians  at  the 
triage  point.  The  step  history  indicates  that  police  effectiveness  is  the  first 
variable  to  enter  the  model,  explaining  about  77%  of  the  variance.  In  the  effect 
test,  this  variable  seems  to  have  a  dramatically  higher  impact  on  the  model  than 
any  other  variable,  shown  by  the  F-test  of  1 ,250,  versus  25  and  20  for  the  next 
two  highest  variables. 
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Figure  31.  Stepwise  Regression  Output  from  JMP,  Response  =  Percentage  of  Civilians  at 

Triage  Point,  Additive  Model 

Figure  32  shows  the  details  of  the  multiple  regression  model  developed. 
The  original  model  is  the  one  located  on  the  left-hand  side  of  the  figure,  an 
additive  (main  effect)  model.  Notice  the  Residual  by  Predicted  Plot;  there 
appears  to  be  a  quadratic  relationship  between  the  effectiveness  of  police  orders 
and  percentage  of  civilians  at  the  triage  point.  After  adding  the  quadratic  term  for 
police  effectiveness  (on  the  right-hand  side  of  the  figure),  notice  that  the  plots  are 
much  more  well  formed,  with  an  R2  of  approximately  90%. 
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Figure  32.  Stepwise  Regression  Plots 
3.  MOE  3.  Neutralization  of  Enemy  Gunman 
a.  Small  FRLH 


Analysis  of  the  third  MOE  follows  the  same  pattern  as  the  first  two. 
There  is  one  difference  in  the  generation  of  this  MOE.  There  are  several 
simulation  runs  in  which  the  police  force  does  not  kill  all  of  the  gunmen.  To 
account  for  this  result,  the  author  generated  a  binary  variable.  A  value  of  1 
means  at  least  one  terrorist  survives,  while  0  means  that  all  are  dead.  As  the 
data  are  grouped  and  the  means  collected,  the  Os  and  Is  are  averaged  to  obtain 
the  proportion  of  those  15  replications  for  which  all  of  the  gunmen  are  killed. 

Figure  33  shows  the  increase  in  R2  with  the  number  of  splits  in  the 
regression  tree.  Note  that  after  the  fourth  split  in  the  tree,  there  is  no  appreciable 
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difference  in  R2.  The  increase  resulting  from  making  four  splits,  versus  five,  is 
only  0.036,  from  0.569  to  0.605. 


R  squared  By  Split 
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Figure  33.  Analysis  of  R2  by  Regression  Tree  Split  (Small  FRLH  Experiment, 
Response  =  Percentage  of  Gunmen  Neutralizations) 

Figure  34  shows  the  regression  tree  that  results  from  splitting  four 
times.  Police  effectiveness  has  importance  in  this  model,  but  it  does  not  show  up 
as  a  factor  until  the  third  level,  or  fourth  split.  Of  the  four  variables  included  in 
this  model,  only  one  is  a  factor  that  a  decision  maker  on  the  first  responder  side 
can  readily  affect. 
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Regression  Tree  on  Small  FRLH  Data  to  Determine  Effectiveness 
in  Neutralizing  the  Gunman  (R2  =  0.57) 


Figure  34.  Regression  Tree  on  Small  FRLH  Data  to  Determine  Percentage  of 

Gunmen  Neutralized 


Number  1  in  Figure  34  shows  the  best  case  scenario:  if  there  are 
less  than  four  gunmen,  we  expect  all  of  the  gunmen  to  be  neutralized 
approximately  90%  of  the  time. 

Number  2  in  Figure  34  illustrates  the  worst  case  scenario.  Below  is 
the  interpretation  of  this  leaf  in  the  decision  tree.  If 

•  there  are  more  than  four  gunman,  and 

•  agitator  vulnerability  is  greater  than  6%,  and 

•  police  marksmanship  is  less  than  0.45,  and 

•  if  agitator  vulnerability  is  less  than  48%, 

then  you  would  expect  all  terrorist  gunmen  to  be  killed  only  about 

half  the  time. 
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Agitator  vulnerability  is  involved  in  two  of  the  four  levels  of  this  tree, 
highlighting  a  possible  secondary  effect  that  is  intuitive.  If  the  agitators  are  less 
vulnerable,  they  draw  the  attention  of  the  first  responders  away  from  the  gunmen. 

b.  Large  FLRH  Experiment 

One  of  the  strengths  of  MAS  is  the  ability  to  “grow”  data  to  analyze 
a  given  problem  in  greater  and  greater  levels  of  detail.  Analysis  of  the 
effectiveness  in  neutralizing  the  terrorist  gunman  is  an  example  of  this  strength. 
The  small  FLRH  experiment  yielded  several  instances  in  which  all  of  the 
terrorist  gunmen  were  not  killed.  To  ensure  accurate  insight  is  gained  into  the 
effectiveness  of  neutralizing  the  gunmen,  more  data  can  be  “grown”  using  the 
principles  of  data  farming.  In  this  case,  the  large  FRLH  experiment  resulted  in  a 
data  set  that  was  more  than  10  times  the  size  of  the  small  experiment 
(45,360  rows  versus  4,320  rows),  enabling  analysis  at  a  finer  level  of  detail. 

Figure  35  shows  the  increase  in  R2  with  the  number  of  splits  in  the 
regression  tree.  Diminishing  returns  in  R2  occurs  after  the  fifth  split.  The 
increase  resulting  from  making  six  splits,  versus  five,  is  only  0.047,  from 
0.534  to  0.581. 


Figure  35.  Analysis  of  R2  by  Regression  Tree  Split  (Large  FRLH  Experiment, 
Response  =  Percentage  of  Gunman  Neutralizations) 
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The  regression  trees  resultant  from  modeling  the  large  and  the 
small  FLRH  have  a  key  difference.  In  the  large  FRLH  experiment,  police  order 
effectiveness  is  the  most  important  factor;  the  single  variable  that  explains  the 
most  variance  (see  Figure  36). 


Figure  36.  Regression  Tree  on  Large  FRLH  Data  to  Determine  Percentage  of 

Gunmen  Neutralized 
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As  illustrated  in  Figure  36,  only  two  factors  (of  a  possible  38)  are 
unique  in  the  first  six  splits.  See  Figure  37  for  a  contour  plot  that  highlights  the 
relationship  between  police  order  effectiveness  and  the  number  of 
terrorist  gunmen  in  determining  if  all  the  gunmen  will  be  killed.  The  coarseness 
of  the  plot  is  explainable  in  part  by  the  variables  representing  the  36  dimensions 
not  captured  on  the  axes.  The  picture  contains  an  overlay  of  the  splits  made  on 
the  regression  tree. 
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Figure  37.  Relationship  of  Police  Effectiveness  in  Giving  Orders  to  the  Number  of 
Terrorist  Gunmen  in  Determining  Percentage  of  Times  Gunmen  Killed 
(Large  FRLH  Experiment) 

(best  viewed  in  color) 

The  relationship  between  the  number  of  gunmen  in  the  simulation 
and  the  difficulty  in  killing  them  all  is  obvious.  It  may  not  be  obvious  why  the 
effectiveness  in  the  police  officers’  use  of  a  paintball  weapon  impacts  the  survival 
of  the  gunmen.  After  viewing  the  simulation  several  times,  a  possible 
explanation  lies  in  the  police  officers’  focus.  Simply  put,  they  do  not  do  two 
things  at  once  very  well.  As  the  police  order  effectiveness  increases,  more 
civilians  move  to  the  triage  points  faster,  resulting  in  the  police  having  the  ability 
to  focus  on  killing  the  gunmen.  Based  on  the  settings  given  to  the  police,  it  is 
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expected  that,  when  more  civilians  are  in  the  area,  the  police  will  be  distracted  by 
the  additional  civilians  and  not  as  responsive  to  the  gunmen. 

4.  MOE  4.  First  Responders  Killed  or  Wounded 

Figure  38  shows  the  increase  in  R2  with  the  number  of  splits  in  the 
regression  tree.  Note  that  after  the  third  split  in  the  tree,  there  is  no  appreciable 
difference  in  R2.  The  increase  resulting  from  making  four  splits,  versus  five,  is 
only  0.035,  from  0.687  to  0.722. 


R  squared  By  Split 
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Figure  38.  Analysis  of  R2  by  Regression  Tree  Split  (Small  FRLH  Experiment, 
Response  =  Proportion  of  First  Responders  Killed  or  Injured) 

Figure  39  shows  the  regression  tree  that  results  from  splitting  three  times. 
Police  effectiveness  is  again  the  most  important  factor  in  predicting  our  response 
variable,  the  proportion  of  first  responders  killed  or  injured  in  the  first  hour. 
Figure  40  shows  a  check  done  with  linear  regression  to  ensure  that  similar 
variables  are  included  with  an  additive  multiple  regression  model  (without 
interactions).  Fortunately,  both  methods  yield  similar  results,  both  in  variable 
selection  and  in  R2;  therefore,  we  can  be  confident  that  we  are  truly  capturing  the 
most  important  variables  in  the  scenario. 
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Regression  Tree  on  Small  FRLH  Data  to  Determine  Proportion  of 
First  Responders  Killed  or  Injured  (R2  =  0.69) 


Figure  39.  Regression  Tree  on  Small  FRLH  Data  to  Determine  Proportion  of 
First  Responders  Killed  or  Injured 


Figure  40.  Results  of  Multiple  Regression  (Stepwise)  Variable  Selection  (Small  FRLH 
Experiment,  Response  =  Proportion  of  First  Responders  Killed  or  Injured) 
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It  is  interesting  that  police  marksmanship,  or  effectiveness  in  giving  orders 
to  civilians,  has  a  dramatic  impact  on  the  proportion  of  first  responders  that  are 
killed  or  injured.  This  type  of  “marksmanship”  is  associated  with  a  paintball 
weapon  that  should  not  directly  affect  first  responders.  Further  inspection  of  the 
scenario  and  the  model  will  determine  whether  police  order  marksmanship  has 
its  effect  directly,  or  through  a  synergistic  effect  or  emergent  behavior  caused  in 
other  agents. 

Figure  41  illustrates  the  relationship  between  vulnerability  of  terrorist 
agitators  and  effectiveness  of  police  in  using  their  paintball  “orders”  weapon.  The 
data  points  are  included  in  this  plot  to  give  the  reader  an  idea  of  the  data  filling 
nature  of  Hernandez’s  FRLH  design. 


Figure  41.  Relationship  of  Police  Effectiveness  in  Giving  Orders  to  Terrorist  Agitator 
Vulnerability  in  Determining  Proportion  of  First  Responders  Killed  or  Injured 

(Small  FRLH  Experiment) 

The  relationship  depicted  in  Figure  41  is  as  one  would  expect,  if  the 
weapon  addressed  with  police  marksmanship  is  a  lethal  weapon.  Because  it  is  a 
paintball  weapon,  it  is  important  to  check  the  model  settings  and  ensure  that 
there  is  no  artifact  in  the  simulation  that  is  causing  the  weapon  to  have  a 
capability  that  the  weapon  should  not  have. 
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This  chapter  was  an  analysis  of  the  data  that  resulted  from  execution  of 
the  simulation  model  developed  in  accordance  with  steps  9  and  10  of  the 
methodology  established  in  Figure  5  of  this  work,  and  run  using  efficient 
experimental  design,  steps  11  and  12.  Given  more  time,  much  more  analysis 
could  be  accomplished  with  this  data.  The  data  analysis  began  with  gaining 
quick  insight  into  the  data,  using  a  small  FRLH  design.  Areas  of  particular 
interest  were  brought  into  further  focus  using  two  more  detailed  approaches,  a 
gridded  design,  and  a  larger,  more  space  filling  Latin  Hypercube  design.  The 
final  chapter  of  this  research  provides  the  conclusions  gained  from  the 
development  of  the  first  response  simulation  methodology  and  analysis  of  the 
simulation  data.  In  addition  to  the  conclusions  of  this  study,  Chapter  VII  includes 
recommendations  for  future  study. 
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VII.  CONCLUSIONS  AND  RECOMMENDATIONS 
FOR  FUTURE  STUDY 


This  chapter  summarizes  the  contributions  of  this  research,  describing 
both  general  methodological  contributions  and  specific  findings  that  relate  to  the 
established  research  questions.  Following  the  discussion  of  conclusions,  the 
author  provides  recommendations  to  government  agencies  for  planning  and 
coordinating  emergency  first  response.  Recommendations  for  future  study 
comprise  the  final  section  of  this  research,  outlining  ways  in  which  this  work  can 
be  carried  forward. 

A.  CONCLUSIONS  AND  CONTRIBUTIONS 
1.  Methodology  Contributions 

a.  Organizational  Learning  Methodology 

Organizational  learning  is  a  process  by  which  an  organization 
deliberately  expands  its  knowledge  base  for  a  certain  purpose.  The  methodology 
proposed  in  Figure  2  of  Chapter  I  illustrates  how  to  use  low  resolution  simulation 
in  conjunction  with  high  resolution  simulation  to  facilitate  the  organizational 
learning  process  during  preparation  for  a  TOPOFF  exercise.  Application  of  this 
methodology  can  certainly  result  in  more  effective  planning,  a  more  focused 
exercise,  and  a  better-trained  response  force.  In  addition,  application  of  this 
methodology  could  result  in  more  efficient  use  of  sparse  resources,  such  as 
training  dollars  and  manpower. 

b.  Emergency  First  Response  MAS  Methodology 

Simulation  of  emergency  first  response  provides  decision  makers 
with  a  tool  that  provides  insights  that  cannot  practically  be  gained  through  other 
means.  The  ability  to  analyze  tens  of  thousands  of  different  situations  and 
variable  combinations  provides  the  analyst  with  the  ability  to  gain  a  much  more 
thorough  knowledge  of  a  problem’s  response  surface  than  can  be  provided  by 
subject  matter  expertise  or  point  estimates,  gained  through  high  resolution 
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simulation.  The  methodology  presented  in  Figure  6  of  Chapter  III  shows  a 
step-by-step  process  that  an  analyst  can  follow  to  use  a  low  resolution  simulation 
to  credibly  analyze  emergency  first  response  to  a  crisis  situation. 

c.  Experimental  Design  Methodology 

Efficient  experimental  design  is  the  bridge  that  enables  the  MAS  to 
be  analyzed  in  several  dimensions,  fully  leveraging  the  power  of  modern 
computers.  The  experimental  design  methodology  developed  in  Chapter  V 
provides  the  analyst  with  a  mechanism  to  take  a  broad  approach  to  initial 
analysis,  using  Hernandez’s  FRLH  designs.  The  analyst  then  narrows  the  focus 
on  areas  of  interest,  using  a  space  filling  NOLH  design,  such  as  those  developed 
by  Cioppa.  Finally,  to  isolate  the  effects  of  a  small  number  of  variables,  the 
analyst  can  utilize  a  gridded  design. 

2.  Analytical  Conclusions 

The  analytical  results  here  demonstrate  a  proof  of  concept.  It  is  possible 
to  gain  useful  insight  through  the  utilization  of  a  low  resolution  model,  such  as  a 
MAS.  Although  it  is  possible  to  provide  decision  support  analysis,  the  findings  in 
this  research  should  not  be  directly  applied  to  the  TOPOFF  process  or  to 
Baltimore  first  responders.  To  provide  analysis  that  is  relevant  to  a  given 
agency,  it  is  imperative  that  accurate  local  force  structure  and  SOPs  be  obtained. 
In  addition,  local  subject  matter  experts  (SMEs)  should  be  involved  in  model 
verification.  When  these  steps  are  taken,  it  is  possible  that  insights  such  as 
those  described  in  Chapter  6  can  be  relevant  to  first  response  organizations. 

Overall,  the  most  important  factor  in  determining  the  effectiveness  of 
first  response  forces  is  the  effectiveness  with  which  the  police  forces  are  able  to 
calm  the  civilians  and  direct  them  to  the  triage  points.  Due  to  the  ROE  used  for 
the  police  in  this  simulation,  they  fixate  on  the  civilian  they  are  assisting,  until  that 
civilian  receives  the  help  he  or  she  needs.  If  the  police  are  effective,  they  can 
then  move  on  to  other  priorities  of  work,  such  as  addressing  an  injured  unit 
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member  or  terrorist  threat.  If  the  police  are  not  effective,  they  do  not  disengage 
and  execute  another  mission. 

a.  What  is  the  Most  Appropriate  Mix  of  Response  Forces? 

This  analysis  provided  insight  into  the  determination  of  response 
force  mix  by  focusing  on  two  fundamental  first  response  MOEs: 

•  Percentage  of  civilians  not  treated  in  the  first  hour 

•  Percentage  of  civilians  relieved  of  panic  in  the  first  hour 

Although  not  able  to  determine  a  true  recommended  mix  of  police 
forces  to  medics  to  firemen,  the  analysis  showed  that  the  number  of  responders 
is  not  as  important  as  their  effectiveness. 

(1)  Number  of  Civilians  Injured  After  the  First  Hour.  This 
MOE  measures  the  effectiveness  of  the  first  responders  in  addressing  the 
civilians’  physical  needs,  i.e.,  first  aid  and  medical  stabilization.  Effective  police 
interpersonal  relationship  skills  with  the  crowd  is  the  most  important  factor  in 
determining  how  many  civilians  are  injured  after  the  “Golden  Hour,”  the  first  hour 
after  an  incident.  At  medium  and  high  levels  of  police  effectiveness,  only  a  few 
injured  civilians  are  able  to  depart  the  first  response  area  of  influence  without 
being  given  first  aid  by  a  policeman  or  treated  by  a  medic.  It  is  only  at  very  low 
levels  of  effectiveness  that  the  police  become  “overwhelmed”  by  the  number  of 
civilians  in  the  area  of  the  bomb  blast  and  are  not  as  effective. 

(2)  Percentage  of  Panic-Relieved  Civilians  After  the  First 
Hour.  This  MOE  measures  the  first  response  force’s  ability  to  address  the 
civilians’  psychological  needs,  i.e.,  the  ability  to  calm  the  crowd  and  restore 
order.  Police  effectiveness  in  maintaining  crowd  control  is  again  the  most 
important  factor  in  addressing  the  needs  of  civilians,  and  clearly  dominates  all  of 
the  other  36  factors  considered  in  the  analysis. 
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b.  What  is  the  Impact  of  Communication  Effectiveness  in 
Emergency  First  Response  to  a  VBIED  in  Baltimore’s 
Inner  Harbor? 

Communication  effectiveness  plays  a  key  role  in  the  ability  of 
command  and  control  elements  to  relay  information  vertically  and  laterally  to 
increase  response  force  situational  awareness  in  the  response  area.  This 
simulation  analyzes  possible  physical  characteristics  of  communication, 
i.e.,  effectiveness  of  a  given  piece  of  equipment.  This  model  does  not  attempt  to 
simulate  the  human  dimension  of  communication,  such  as  measuring  the 
equivalence  of  the  message  sent  versus  the  message  received. 

(1)  Neutralization  of  Terrorist  Gunmen.  In  this  simulation,  a 
key  feature  is  the  passing  of  information  from  the  first  responders  to  an  incident 
command  cell.  The  incident  command  cell  then  processes  and  laterally  passes 
the  information  to  other  command  cell  members;  the  consolidated  information  is 
disseminated  to  unit  members.  One  such  piece  of  information  is  the  location  of 
terrorist  gunmen.  The  analysis  showed  that  communication  was  not  a  key  factor 
in  the  neutralization  of  the  enemy  gunmen;  rather,  police  effectiveness  in  giving 
orders  to  civilians  and  the  number  of  gunmen  were  the  dominant  factors. 

A  possible  explanation  exists  for  the  counterintuitive 
importance  of  police  orders.  Importance  of  orders  may  lie  in  the  ROE.  In  this 
simulation,  police  have  a  higher  desire  to  help  the  civilians  than  kill  the  enemy. 
As  such,  with  decreased  police  effectiveness,  the  civilians  are  not  helped  as 
quickly.  The  police,  engaged  in  trying  to  assist  the  civilians  with  decreased 
effectiveness,  do  not  disengage  from  the  civilians  in  order  to  address  the  threat 
of  gunmen. 

The  lack  of  importance  of  communication  may  result  from 
the  procedural  nature  of  first  response.  Effective  first  response  involves  effective 
execution  of  SOPs.  Communication,  while  important  to  maintain  situational 
awareness,  may  not  be  a  key  factor  in  first  response.  Agencies  know  what  to  do 
when  they  arrive  on  scene,  most  likely  have  trained  on  it,  and  are  prepared  to 
execute  in  accordance  with  local  SOPs.  Communication  becomes  of  paramount 
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importance  in  the  execution  of  the  follow-on  mission,  as  police,  fire,  and  medical 
forces  arrive  from  outside  the  crisis  area  (not  modeled  in  this  simulation).  The 
incident  command  post  must  dispatch  these  units  to  the  appropriate  area, 
requiring  communication  and  situational  awareness.  This  simulation  ends  before 
these  additional  follow-on  forces  arrive,  a  possible  reason  why  communication 
does  not  appear  as  a  relevant  factor  in  this  analysis. 

(2)  Proportion  of  First  Responders  Killed  or  Wounded.  The 
proportion  of  first  responders  killed  or  injured  relates  to  the  importance  of 
communication  in  a  similar  manner  to  the  neutralization  of  the  terrorist  gunmen, 
but  adds  an  additional  dimension.  Police  must  neutralize  the  lethal  threat  of  the 
gunmen,  or  incur  additional  casualties.  All  first  responders  must  be  able  to 
communicate  their  injury  status  to  unit  members  to  be  able  to  receive 
medical  attention. 

Communication  does  not  appear  to  be  a  relevant  factor  in 
determining  the  proportion  of  first  responders  killed  or  injured.  Rather,  the  most 
important  factor  is  police  effectiveness  in  addressing  civilian  needs.  As 
suggested  in  section  VLB.  1  .b,  this  may  have  been  the  result  of  modeling  SOPs. 
Again,  a  model  could  easily  be  adjusted  to  account  for  lack  of  SOPs, 
undisciplined  following  of  SOPs,  or  SOPs  that  cannot  be  followed  because  of  the 
situation.  In  these  cases,  analysis  may  reveal  a  higher  level  of  importance  for 
communication  factors. 

B.  RECOMMENDATIONS  FOR  EMERGENCY  PREPAREDNESS 

Simulation  is  a  valuable  tool  that  first  response  agencies  could  use  to 
facilitate  emergency  preparedness  in  two  ways: 

•  Training  of  leaders  and  staff 

•  Exercise  planning 

1.  Training  of  Leaders  and  Staff 

Emergency  response  organizations  at  virtually  all  levels  should  use 
simulation  as  part  of  their  training  program  for  leaders  and  staff.  Planning, 
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executing,  and  analyzing  a  simulation  provides  a  forum  that  helps  leaders  and 
staff  better  understand  local  SOPs.  Executing  a  credible  simulation  requires  the 
simulation  to  emulate  reality.  To  prepare  the  simulation,  the  modeler  must 
thoroughly  understand  what  the  SOP  says,  but  also  what  it  means:  the  effect 
meant  to  be  achieved  by  a  certain  directive.  During  the  execution  of  the 
simulation,  the  training  audience  may  see  emergent  behavior  by  agents  that 
causes  them  to  question  and  improve  certain  facets  of  the  SOP.  During  the 
analysis  of  the  data  generated  by  the  simulation,  the  leaders  and  staff  may 
encounter  surprises  in  the  data  that  also  causes  the  reexamination  of  portions  of 
the  SOP. 

2.  Exercise  Planning 

Planners  can  make  large-scale  exercises  more  effective  training  tools 
through  the  use  of  the  organizational  learning  process  that  incorporates  the  MAS 
modeling  methodology  established  by  this  research.  Through  the  development 
and  execution  of  simulations,  leaders  and  staff  develop  an  understanding  of  the 
given  problem  that  is  simply  not  gained  through  less  structured  brainstorming 
and  planning  sessions. 

C.  RECOMMENDATIONS  FOR  FUTURE  STUDY 

Analysis  of  a  given  problem  can  result  in  the  development  of  as  many  or 
more  questions  than  it  solves.  This  analysis  is  no  exception.  The  author  used  a 
very  specific  set  of  assumptions  to  complete  this  simulation.  The  simulation  and 
the  subsequent  analysis  can  be  considered  one  data  point.  Many  more  data 
points  are  needed  to  better  understand  the  full  capability  represented  in 
this  research. 

This  research  included  a  cursory  qualitative  comparison  of  the  MAS 
developed  to  a  well-established,  high  resolution  model,  EPiCS.  A  future  project 
could  be  the  development  of  a  methodology  to  quantitatively  assess  the 
calibration  of  this  model  to  EPiCS.  The  work  could  then  generalize  the 
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comparison  methodology  to  include  the  comparison  of  other  MAS  simulations  to 
appropriate  high  resolution  simulations. 

In  addition  to  being  calibrated  by  another  model,  this  model  can  be 
calibrated  using  actual  events.  With  the  assistance  of  the  DHS  or  the 
City  of  New  London,  this  model  could  be  populated  with  actual  force  structure 
and  SOPs  associated  with  the  response  forces  for  TOPOFF  3.  The  model  could 
then  be  run  and  analyzed  to  determine  how  close  it  came  to  replicating  the  actual 
events  of  TOPOFF  3  in  New  London. 

Typically  in  crisis  situations,  the  affected  civilians  react  in  different  ways. 
Future  work  could  involve  the  representation  of  civilians  that  assist  emergency 
responders  in  the  accomplishment  of  their  mission.  In  addition,  it  is  possible  to 
model  civilians  that  work  against  first  response  forces,  e.g.,  by  looting  or  even 
attacking  first  responders.  Interaction  with  helpful  civilians  or  antagonistic 
civilians  may  sway  a  neutral  civilian  to  one  side  or  the  other.  The  author  led 
preliminary  work  in  this  area  during  the  12th  Project  Albert  International  Workshop 
(PAIW),  held  in  Boppard,  Germany.  See  Appendix  D  (PAIW  -  12  Findings)  for 
details. 

This  simulation  involved  the  execution  of  one  set  of  first  response  and 
enemy  courses  of  action.  Follow-on  work  could  examine  the  execution  of 
different  courses  of  action  for  both  first  response  and  terrorists.  It  would  be 
interesting  to  note  first  response  courses  of  action  that  are  most  robust  against  a 
variety  of  terrorist  courses  of  action. 

Another  possible  project  that  could  spring  from  this  work  is  analyzing  the 
usefulness  of  separate  vignettes  from  the  model  in  answering  research 
questions.  It  is  possible  that  some  research  questions  do  not  require  the 
aggregation  of  all  agents  and  agencies.  Disaggregation  of  the  model  could  result 
in  much  lower  processing  times  for  simulation  runs,  meaning  a  quicker  turn 
around  of  data  for  analysis.  While  losing  the  synergistic  effects  of  including  all 
agencies  in  the  model,  disaggregation  may  be  appropriate  in 
some  circumstances. 
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The  Pythagoras  scenario  files,  Excel  modeling  files,  and 
electronic  references  are  available  by  contacting  MAJ  Jonathan 
United  States  Military  Academy,  Department  of  Mathematical 
West  Point,  New  York,  or  by  e-mail  at  ionathan.roqinski@us.armv.mil. 
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APPENDIX  A  -  MODEL  IMPLEMENTATION 


This  appendix  provides  detailed  information  about  the  numbers  the  author 
used  in  this  research.  Included  are  details  about  how  distances  were  measured 
for  the  scale  of  the  simulation,  terrain  settings,  weapon  characteristics,  and  side 
affiliations  of  agent  classes.  The  purpose  of  this  appendix  is  to  make  this 
simulation  as  transparent  as  possible,  to  reveal  what  is  going  on 
“under  the  hood.” 

A.  OVERALL  MODEL  CONFIGURATION 
1.  Scaling 

Figure  42  is  a  terrain  snapshot  of  the  Celebrate  Baltimore  area,  taken 
using  Google  Earth,  centered  on  the  Amphitheater  in  Baltimore’s  Inner  Harbor. 


Figure  42:  Terrain  Snapshot  of  Celebrate  Baltimore  Area66 


66lmage  downloaded  from  the  World  Wide  Web  using  Google  Earth  on  12  April  2006. 
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To  measure  the  actual  size  of  this  representation,  the  author  used  the 
latitude  and  longitude  measurements  at  the  four  corners  of  the  picture  and 
entered  them  into  a  latitude/longitude  to  feet  distance  converter  found  at: 
http://www.epa.qov/athens/learn2model/part-two/onsite/ll-dms.htm. 

See  Figure  43  for  the  width  of  the  above  snapshot  and  Figure  44  for 
the  height. 


EPA  On-line  Tools  for  Site  Assessment  Calculation 

Recent  Additions  1  Contact  Us  1  Print  Version  |  E3 

EPA  Home  >  >  Ecosvstems  Research  >  Modelina  Subsurface  Petroleum  Hydrocarbon  Transoort  >  OnSite  on-line  calculator 

Lat-Long  (degrees-minutes-seconds)  to  distance 

Module  Home  Objectives  Table  of  Contents  Previous  <  Next  > 

Distance  from  Latitude  and  Longitudes  (degrees-minutes-seconds) 

Example  Data  |  Calculate  |  Clear  | 

Location  Name  Latitude  Longitude 

|39  |l  7  |11  |76  |37  | 

|39  |17  |11  |76  |36  |22 

|2983.97  |  feet ^J 

Same  calculation  in  decimal  latitude  and  lonaitude. 

Note:  This  calculation  is  not  accurate  enough  for  navigation. 

The  calculations  assume  the  Earth  is  a  perfect  sphere  (which  it  is  not).  These 
results  are  intended  for  use  with  GPS  receivers  used  to  locate  features  on 
leaking  underground  storage  tank  sites.  Check  the  accuracy  of  your  GPS 
receiver  as  large  errors  occur  when  used  for  small  scale  measurements. 

Go  Back  | 

Figure  43:  Calculation  of  Terrain  Box  Width 


Lat-Long  (degrees-minutes-seconds)  to  distance 

Module  Home  Objectives  Table  of  Contents  Previous  <  Next  > 


Distance  from  Latitude  and  Longitudes  (degrees-minutes-seconds) 


Example  Data  | 
Location  Name 


Calculate  | 
Latitude 

[39  fl6  [58 

[39  [i?  [22 


Clear  | 
Longitude 


[?B  [36  \aT 

[?6  [36  [49"" 


|2439.99  |  feet 


Same  calculation  in  decimal  latitude  and  longitude. 

Note:  This  calculation  is  not  accurate  enough  for  navigation. 

The  calculations  assume  the  Earth  is  a  perfect  sphere  (which  it  is  not).  These 
results  are  intended  for  use  with  GPS  receivers  used  to  locate  features  on 
leaking  underground  storage  tank  sites.  Check  the  accuracy  of  your  GPS 
receiver  as  large  errors  occur  when  used  for  small  scale  measurements. 

Go  Back  I 


Figure  44:  Calculation  of  Terrain  Box  Height 


130 


The  goal  of  achieving  a  2,000-foot  by  2,000-foot  square  box  would  be 
achieved  by  making  a  smaller  box  that  is  approximately  67%  of  the  current  box’s 
width  and  approximately  80%  of  its  height. 

2.  Terrain 

Figure  45  illustrates  the  selection  concealment  factor  and  selection  of 
ceiling  height.  The  modeler  decides  on  a  concealment  factor  that  guards  against 
detection  in  the  three  sensing  spectra.  These  factors  are  located  in  the  GUI  on  a 
sliding  scale,  from  0.0  to  1.0  (least  visible  to  most  visible).  Movement  and 
protection  factor  are  located  below  concealment,  and  are  scaled  in  the  same 
manner  as  concealment.  A  movement  factor  of  0.0  means  that  an  agent  cannot 
move  within  this  terrain  feature,  while  a  factor  of  1.0  results  in  unencumbered 
movement.  A  protection  factor  of  1.0  results  in  total  protection,  while  0.0  results 
in  no  protection. 


Figure  45:  Instantiating  Terrain  Factors 

Figure  46  shows  the  Pythagoras  representation  of  the  terrain  located  in 
Figure  42. 
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Figure  46.  Pythagoras  Representation  of  Celebrate  Baltimore  Area 
3.  Weapons 

Figures  47-49  show  the  weapon  settings  used  in  this  simulation.  In 
Figure  47,  fire  rate  per  minute  and  engagement  range  each  have  columns  that 
represent  the  actual  scale  values  and  their  corresponding  values  as  scaled  for 
this  Pythagoras  simulation.  The  workbook  used  to  track  weapon  characteristics 
was  adapted  from  research  completed  by  Major  Michael  Babilot,  USMC.67 


67Michael  Babilot,  “Comparison  of  a  Distributed  Operations  Force  to  a  Traditional  Force  in 
Urban  Combat,”  Masters  Thesis,  Naval  Postgraduate  School,  Monterey,  CA,  September  2005. 


132 


General  Weapon  Characteristics 
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Figure  47.  General  Weapon  Characteristics 

Figure  48  shows  weapon  paintball  characteristics  assigned  by  the  author 
for  this  scenario. 


Paintball  Characteristics 

Delta 

Delta  | 

Type 

Red  |  Green  |Blue 

Alpha 

Beta  |  Gamma 

M4 

M9 

AK47 

Agitator 

10 

Bomb  (Carleton) 

20 

Bomb  (Cookie  Cutter) 

20 

Medical  Kit 

-10 

Orders  -  From  Firemen 

-10 

Orders  -  From  Medics 

-10 

Orders  -  From  Police 

-10 

Figure  48.  Paintball  Weapon  Characteristics 

Figure  49  represents  probability  of  hit  data  entered  for  the  weapons  in 
the  model. 
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Figure  49.  Probability  of  Hit  Data 

Figure  50  shows  the  probabilities  of  kill  (for  lethal  weapons)  or 
probabilities  of  restoration  (restorative  weapon),  given  that  targets  are  hit  in  a 
certain  engagement.  This  is  different  than  SSPK.  SSPK  is  calculated  for  a 
certain  weapon,  at  a  range  by  multiplying  the  numbers  given  in  Figures  49  and 
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50.  If  the  range  is  not  shown  in  the  figures,  use  linear  interpolation  between  the 
next  highest  and  next  lowest  number. 


Probability  of  Kill  Given  Hit 
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Figure  50.  Probability  of  Kill,  Given  Hit  Data 

4.  Agent  Side  Property 

Figures  51  and  52  provide  detailed  information  about  the  side  affiliations 
of  each  agent  class  instantiated  in  this  simulation,  using  a  spreadsheet  tool 
developed  by  Northrup  Grumman. 
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Figure  51.  Color  Definitions  of  Each  Agent  Class 
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Figure  52.  Side  Affiliations  of  Each  Agent  Class 
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Movement  Rates 


Figure  53  is  a  screen  shot  of  an  Excel  spreadsheet  movement  calculator 
adapted  from  work  done  by  Babilot.68 


Figure  53.  Excel  Movement  Rate  Calculator 


68Michael  Babilot,  “Comparison  of  a  Distributed  Operations  Force  to  a  Traditional  Force  in 
Urban  Combat,”  Masters  Thesis,  Naval  Postgraduate  School,  Monterey,  CA,  September  2005. 
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This  appendix  served  to  answer  questions  the  reader  may  have  had  about 
the  numbers  used  to  develop  this  simulation  by  describing  where  the  data  came 
from  and  how  the  numbers  were  developed. 
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APPENDIX  B  -  EXPERIMENTAL  DESIGN  METHODOLOGY 


This  appendix  provides  additional  detail  about  the  procedures  used  to 
develop  the  experimental  designs  for  this  research.  It  expands  upon  the 
discussion  of  OLHs,  NOLHs,  and  FRLHs,  to  provide  more  information  on  how  the 
number  of  experimental  design  points  is  calculated  for  a  given  simulation.  In 
addition,  this  appendix  provides  the  Visual  Basic  code  that  can  form  the  bridge 
between  any  Latin  Hypercube  builder  and  study  file  for  Pythagoras  or  MANA. 

A.  DESIGN  POINT  CALCULATIONS 
1.  Design  by  Ye 

Figure  54  shows  the  relationship  between  number  of  variables  and 
required  levels  to  maintain  orthogonality  of  design  points,  as  determined  by  Ye. 


Step  1:  Determine  number  of  variables  ( k ) 

Step  2:  Calculate  m  = 

2 

Step  3:  Calcuate  number  of  required  levels  n  =  2m  +1 
Figure  54.  Relationship  of  k,  m,  and  n,  as  determined  by  Ye 

2.  Design  by  Cioppa 

Table  23  illustrates  a  finding  by  Cioppa  that 

.  .  .  as  the  number  of  levels  doubles  (less  one  for  the  center  point), 
Ye’s  [Orthogonal  Latin  Hypercube]  OLHC  designs  are  able  to 
accommodate  exactly  two  more  variables.69  In  the  new  designs  [by 
Cioppa],  the  corresponding  maximum  number  of  variables 
increases  in  accordance  with  the  following  formula.70 


69K.Q.  Ye,  “Orthogonal  Column  Latin  Hypercubes  and  their  Application  in  Computer 
Experiments,”  Journal  of  the  American  Statistical  Association  -  Theory  and  Methods ,  Vol.  94, 
No.  144,  pp.  1430-1439,  December  1998. 

70Thomas  M.  Cioppa,  “Efficient  Nearly  Orthogonal  and  Space-Filling  Experimental  Designs 
for  High-Dimensional  Complex  Models,”  Ph.D.  Dissertation,  Operations  Research  Department, 
Naval  Postgraduate  School,  Monterey,  CA,  2002. 


139 


k  =  m  + 


r  ^ 


m-1 


Figure  55.  Determination  of  Maximum  Variables  for  Given  Number  of  Design  Points 

(Cioppa) 


Total  Number  of 
Levels  for  Each 
Variable  (n) 

m 

Maximum  Number  of 
Variables  Using 
Cioppa’s  Design  (k) 

Maximum  Number  of 
Variables  Using 

Ye’s  Design 

17 

4 
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6 

33 

5 

11 

8 

65 

6 

16 

10 

129 

7 

22 

12 

Table  22.  A  Comparison  Illustrating  the  Increased  Number  of  Variables  that  can  be 
Examined  by  Extending  Ye’s  (1998)  Construction  Algorithm  for  OLHCs 


3.  Design  by  Hernandez 

Hernandez71  found  that  that  as  the  number  of  levels  increases,  the 
possible  number  of  variables  that  can  be  explored  increases  to  no  more  than 
one-third  the  number  of  levels.  Hernandez  found  that  using  correlation  reductin 
methods,  it  is  possible  to  create  a  design  that  meets  Cioppa’s  NOLH  criteria, 
when  the  number  of  design  points  is  greater  than  or  equal  to  49  (see  Figure  56). 


n  :=  number  of  design  points  (levels) 
k  :=  number  of  variables 

When  n  >  49, 

n>3k 


Figure  56.  Relationship  Between  Number  of  Variables  and  Number  of  Levels  in 
Hernandez’s  Design  of  Experiments 


71Alejandro  D.  Hernandez,  “Expanding  the  Family  of  Efficient,  High-Dimensional,  Nearly 
Orthogonal  Experimental  Designs  through  Flexible  Random  Latin  Hypercubes  Methods,” 
Ph.D.  Dissertation,  Operations  Research  Department,  Naval  Postgraduate  School,  Monterey,  CA, 
pending  publication. 
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Table  24  highlights  the  difference  between  Cioppa’s  design  and 
Hernandez’s  design  in  the  number  of  possible  variable  analyzed  in  a  given 


number  of  possible  design  points. 


Total  Number  of 
Levels  for  Each 
Variable  (n) 

m 

Maximum  Number  of 
Variables  Using 
Hernandez’s  Design  (k) 

Maximum  Number  of 
Variables  Using 
Cioppa’s  Design 

129 

7 

43 

22 

257 

8 

85 

29 

513 

9 

171 

37 

1,025 

10 

341 

46 

2,049 

11 

683 

56 

Table  23.  A  Comparison  Illustrating  the  Increased  Number  of  Variables  that  can  be 
Examined  by  Extending  Cioppa’s  Construction  Algorithm  for  NOLHs 

B.  EXCEL  MACRO  BY  MICHEL 

Figure  57  is  Visual  Basic  code,  developed  by  Major  Christopher  Michel, 
USMC.72  This  code  was  developed  to  run  in  Microsoft  Excel;  it  converts  a 
Latin  Hypercube  into  a  format  that  easily  imported  into  a  study  file  to  execute  a 
data  farming  experiment  in  either  Pythagoras  or  MANA. 


72Christopher  Michel,  “Supporting  A  Marine  Air  Ground  Task  Force  With  Appropriate 
Quantities  of  Ground  Based  Fire  Support,”  Masters  Thesis,  Naval  Postgraduate  School, 
Monterey,  CA,  September  2006. 
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Sub  TransposeNuitiberHacro  [ ) 

i 

1  TransposeNumberMacro  Macro 
1  Macro  recorded  3/27/2006  by  localadmincm 


'Ensure  data  starts  in  2nd  column  to  use  this  code 
xX  =  0 

For  DesignPoint  =  0  To  1 44 

For  ]  =  1  To  48  1  factors 

Sheets  ( rr  Y  OUR_L  AT  IN_H  YP  E  RC  UB  E_G0E  S_HE  RE rr )  .Select 

Cells  (2  +  DesignPoint,  j  +  1)  .Copy 

Sheets ( "VBA  for  XML") . Select 

Cells (xX  +  j  +  1,  2). Select 

ActiveSheet . Paste 

Application. CutCopyMode  =  False 

Next  j 

xX  =  xX  +  50  1  transposes  data 

Next  DesignPoint 
End  Sub 


Figure  57.  Excel  Macro  by  Michel 
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APPENDIX  C  -  DATA  ANALYSIS 


This  appendix  provides  an  illustration  of  the  Clementine  stream  used  to 
clean  the  data  resulting  from  the  simulation  runs  conducted. 


A. 


LARGE  FRLH  EXPERIMENT 


Figure  58  Illustrates  the  process  by  which  Clementine  can  be  used  to 
transform  the  data  received  from  a  Pythagoras  submission  to  MHPCC  into  a 
data  set  that  contains  exactly  the  information  required  for  analysis. 


Large  Experiment  in  Clementine 


^^“Cleaned”  data  exported  into 
separate  file 


Three  files  are 
joined  into 


Figure  58.  Using  Clementine  to  Clean  Large  FRLH  Data 
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B.  SMALL  FRLH  EXPERIMENT 


The  same  Clementine  stream  is  used  to  clean  the  small  data  set  as  was 
used  for  the  large.  The  input  files  are  simply  deleted,  and  the  files  corresponding 
to  the  small  FRLH  data  are  uploaded. 

C.  GRIDDED  EXPERIMENT 

The  gridded  experiment  required  the  same  approach  to  the  FRLH 
experiments,  but  required  the  addition  of  two  additional  file  inputs.  The  existing 
streams  were  simply  copied  and  pasted;  it  was  not  necessary  to  build  them 
from  the  start. 
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APPENDIX  D  -  PAIW-12  FINDINGS 


This  appendix  provides  information  about  findings  that  resulted  from  the 
PAIW-12  workshop,  held  in  Boppard,  Germany,  from  June  1-10,  2006.  The  work 
of  PAIW-12  Syndicate  2  was  a  follow  on  work  from  the  research  presented  in  this 
document.  Syndicate  2  was  composed  of  the  following  people: 

•  Major  Jonathan  W.  Roginski,  United  States  Army 

•  Ms.  Gudrun  Wagner,  Germany,  European  Aeronautic  Defence  and 
Space  (EADS)  Systems,  Defence  and  Security 

•  Mr.  Ole  Jakob  Sendstad,  Norway,  Norwegian  Defence 
Research  Establishment 

The  overall  goal  of  the  workshop  for  Syndicate  2  was  to  model  pro  and 
anti  first  responder  civilians,  and  the  effect  of  neutral  civilians  adopting  an 
allegiance  of  either  pro  or  anti  first  response,  while  decreasing  simulation  run 
time.  The  syndicate  used  the  following  MOEs  to  evaluate  first  responder 
success;  all  statistics  were  collected  at  the  end  of  the  simulation  run. 

•  Percentage  of  civilians  killed  or  injured 

•  Percentage  of  civilians  near  the  triage  point 

Figure  59  is  an  overview  of  what  was  accomplished  during  the  week. 
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What  was  Accomplished 


Simulation  pared  down  from  40+  minutes  to  8-12  minute 
run  time 


2000+  MHPCC  simulation  runs  to  analyze  impact  of 
civilian  numbers 

-  38  variables 

-  144  design  points  (15  replicates) 


Flexible  Random 
Latin  Hypercube 


-  MOE  1 :  Percentage  of  Civilians  Killed  or  Injured 

-  MOE  2:  Percentage  of  Civilians  near  Triage  Point 


Figure  59.  PAIW-12  Accomplishments 

The  syndicate  was  able  to  complete  the  modeling  of  pro  and  anti  response 
civilians,  in  addition  to  neutral  civilian  transitions  in  less  than  one  week,  but  was 
not  able  to  have  the  data  set  returned  from  MHPCC  in  time  to  analyze  before  the 
close  of  the  workshop. 

A.  MOE  1 

The  syndicate  did  receive  the  base  case  scenario  submission  in  time  for 
analysis,  resulting  in  some  interesting  results.  Figure  60  is  an  illustration  of 
analysis  results  that  focused  on  MOE  1  from  Figure  59. 
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Contour  Plot  for  Percent_lnjured_or_Dead 

l.l—i -  — Legend- 


-o-1  H - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 

-1  012345678 

Traffic  1  instance 


Figure  60.  MOE  1  Results 

The  CART  diagram  (not  shown)  indicated  that  the  two  most  important 
variables  in  determining  the  percentage  of  civilians  killed  or  injured  were  the 
number  of  police  at  intersection  1  (northwest  corner)  and  the  lethality  of  the 
bomb.  The  contour  plot  in  Figure  60  plots  the  two  most  important  variables  on 
the  horizontal  and  vertical  axes  and  uses  color  to  emphasize  the  response 
variable:  civilians  injured  or  killed. 

Analysis  of  MOE  1  seems  to  indicate  that,  while  the  number  of  police  at 
this  location  is  important,  more  is  not  always  better.  In  this  simulation,  there 
appears  to  be  diminishing  returns  after  three  police  arrive  at  the  location;  there  is 
little  increase  in  the  MOE  with  additional  police  at  the  location.  Although  a 
low  resolution  simulation  cannot  be  trusted  to  determine  that  the  number  3  is 
exactly  the  correct  number  at  which  the  diminishing  returns  happens,  it  can  be  an 
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indicator  that  the  point  of  diminishing  returns  exists  and  that  the  decision  maker 
should  be  aware  of  it. 

B.  MOE  2 

Using  a  similar  technique  to  that  used  for  MOE  1 ,  the  syndicate  first  used 
a  CART  diagram  to  identify  the  most  important  variables.  For  this  MOE,  the  most 
important  variables  were  the  lethality  of  the  bomb,  and  the  number  of  police  that 
patrol  the  middle  patrol  area.  The  contour  plot  in  Figure  61  plots  the  two  most 
important  variables  on  the  horizontal  and  vertical  axes  and  uses  color  to 
emphasize  the  response  variable:  civilians  injured  or  killed. 


Contour  Plot  for  Percent_Near_Triage_Point 


Figure  61.  MOE  2  Results 

Analysis  of  MOE  1  seems  to  indicate  a  similar  diminishing  return  in  the 
central  patrol  area  as  was  indicated  at  the  northwest  intersection  in  MOE  1, 
although  the  relationship  is  not  as  strong. 
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Overall,  analysis  of  the  data  resulting  from  PAIW-12  suggests: 

•  It  is  possible  to  create  the  forces  in  an  emergency  response 
simulation,  debug  the  scenario,  and  use  data  farming  techniques  to 
receive  data  to  analyze  within  one  week. 

•  There  may  be  an  optimal  number  of  traffic  police  per  intersection 
and  patrolmen  per  area.  Exceeding  that  number  could  result  in 
diminishing  returns,  or  an  inefficient  use  of  manpower. 
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